DualWorldBench

What is DualWorldBench?

A benchmark for executable pickup-and-delivery planning.

DualWorldBench evaluates agent planning under controlled symbolic-temporal and grounded-topological complexity. The task interface is shared: an agent receives task specifications and, when applicable, a grounded scene, then outputs a sequential plan composed of move, pickup, and deliver actions.

DualWorldBench task overview with task specification, SimWorld scene, and sequential action plan. — The agent must produce a topology-aware action plan while satisfying pickup-before-delivery, capacity, and task-matching constraints.

Why is this problem important?

Delivery planning exposes two missing capabilities in current benchmarks.

01

Real agents face coupled combinatorial constraints.

A delivery agent must jointly handle task order, rewards, deadlines, pickup-before-delivery precedence, capacity limits, and travel cost. This makes planning difficulty grow at the sequence level rather than as isolated subtasks, but many existing evaluations still use simplified settings with weak temporal and combinatorial pressure.

02

Planning must integrate multi-source evidence.

In realistic scenes, symbolic task descriptions and visual topology are not trivially aligned: two locations close in coordinates may be far apart in reachable distance. Existing benchmarks rarely isolate whether agents can combine perception with structured constraints to produce actionable plans.

What do we do?

We decompose planning into TextWorld, SimWorld, and DualWorld-VQA.

The benchmark is integrated because it uses a shared pickup-and-delivery interface, and decomposable because each component isolates a different source of difficulty.

TextWorld

Procedurally generated symbolic pickup-delivery tasks with task composition, deadlines, rewards, capacity constraints, and online task release.

SimWorld

Grounded scene layouts evaluated under text, image, and multimodal inputs across open, T1, T2, indoor, and campus scene groups.

DualWorld-VQA

Hierarchical diagnostic questions for visual perception, spatial topology, and logical planning under pickup-delivery constraints.

Evaluation Results

Three experimental questions.

Q1 / Q2 / Q4: How do planners scale with combinatorial difficulty?

TextWorld varies task count, task composition, and online release; SimWorld varies task cardinality under grounded scenes.

Q3: Can MLLMs align textual and visual evidence into feasible plans?

SimWorld compares text-only, image-only, and multimodal inputs under the same executable interface.

Q5: Are grounded plans robust across scene topology?

Scene groups test whether agents can transfer from open layouts to corridor, indoor, and campus-scale topology.

TextWorld: symbolic-temporal planning

TextWorld reports cumulative reward R and execution time T. Higher reward is better; lower execution time is better.

Table 1. TextWorld results under varying task counts.

Method	Rank	Average		n=5		n=8		n=10		n=12		n=15
Method	Rank	R	T	R	T	R	T	R	T	R	T	R	T
Open-source LLMs
Qwen3-VL-8B	16	68.81	3305.46	19.22	1449.30	121.67	4481.61	35.69	2902.33	135.69	4653.41	31.79	3040.67
Qwen3-VL-30B	10	227.77	5455.32	105.26	2362.78	187.88	5808.95	272.20	5059.84	259.22	6045.60	314.29	7999.41
Llama-2-13B	12	210.18	5713.06	100.28	2396.66	209.73	5471.45	191.57	6393.21	298.18	5912.93	251.15	8391.05
Llama-3-8B	17	19.33	3223.40	26.14	1640.78	13.94	3526.92	50.24	3609.83	6.33	3237.74	0.00	4101.73
Proprietary LLMs
DeepSeek-v3.2	9	239.92	4910.28	107.57	2228.91	204.91	5405.18	212.87	4058.73	310.66	5655.99	363.57	7202.59
Claude-4.0-Sonnet	15	178.74	4347.34	82.63	2057.43	196.58	5054.75	225.41	4328.95	210.74	4221.69	178.32	6073.89
Claude-4.5-Haiku	13	207.92	5907.85	68.54	4083.22	131.19	7037.36	266.26	4818.51	194.89	6708.56	378.69	6891.58
Claude-4.6-Sonnet	14	196.58	4655.09	92.13	2038.50	156.15	5114.04	213.94	4890.29	238.57	4878.23	282.13	6354.39
Gemini-2.5-flash	5	251.11	4353.31	111.10	1875.90	213.24	5118.39	279.18	4175.72	333.12	4934.41	318.88	5662.13
Gemini-3-flash	11	227.74	5377.65	114.77	1829.96	139.42	7173.83	278.53	4445.07	213.96	6802.75	392.01	6636.65
GPT-4o	6	251.05	5245.35	114.77	2226.95	189.94	5808.08	272.78	4805.22	303.95	5662.43	373.82	7724.05
GPT-5	8	244.57	5295.97	114.77	2059.74	213.17	5548.78	179.67	5945.35	323.13	5648.90	392.14	7277.08
GPT-5.4-mini	7	246.81	5207.10	111.10	2219.29	198.84	5532.03	248.54	4925.06	321.70	5662.36	353.88	7696.76
GPT-5.4	4	255.59	4783.51	108.56	2062.11	200.54	5571.86	271.62	4160.17	310.55	5390.21	386.70	6733.20
Combinatorial Optimization Solvers
DFS	3	264.33	4702.45	114.77	2028.71	213.24	5349.95	278.53	4344.86	323.13	5186.65	392.01	6602.05
Online DP	2	264.36	4083.25	114.77	1758.71	213.24	4742.61	278.53	3806.96	323.13	4479.11	392.14	5628.85
Offline DP	1	278.17	4015.28	126.20	1718.67	230.27	4602.62	290.00	3794.25	334.65	4406.58	409.71	5554.29

Table 2. TextWorld results at task count 15.

Method	Rank	Average		A-L1		M-L1		A-L2		M-L2
Method	Rank	R	T	R	T	R	T	R	T	R	T
Open-source LLMs
Qwen3-VL-8B	16	31.79	3040.67	10.04	453.02	60.08	455.02	10.04	4102.59	47.00	7152.04
Qwen3-VL-30B	12	314.29	7999.41	302.97	6359.43	558.94	6800.48	202.29	10685.64	192.96	8152.10
Llama-2-13B	14	251.15	8391.05	302.97	6165.02	558.94	6820.17	142.68	10580.02	error
Llama-3-8B	17	0.00	4101.73	0.00	4.00	0.00	4.00	0.00	9246.89	0.00	7152.04
Proprietary LLMs
DeepSeek-v3.2	9	363.57	7202.59	302.97	5468.36	558.94	5332.65	262.07	9619.76	330.32	8389.61
Claude-4.0-Sonnet	15	178.32	6073.89	136.82	1960.12	378.67	2308.82	197.81	10027.60	error
Claude-4.5-Haiku	7	378.69	6891.58	302.97	4877.98	558.94	5535.86	262.07	9619.76	390.80	7532.73
Claude-4.6-Sonnet	13	282.13	6354.39	302.97	5159.79	362.96	2282.45	224.33	9246.89	238.25	8728.44
Gemini-2.5-flash	11	318.88	5662.13	10.43	450.18	558.94	4649.88	289.21	9619.76	416.92	7928.71
Gemini-3-flash	5	392.01	6636.65	302.97	3615.95	558.94	5613.23	289.21	9619.76	416.92	7697.67
GPT-4o	8	373.82	7724.05	302.97	6020.76	558.94	6346.16	241.98	10273.35	391.38	8255.92
GPT-5	3	392.14	7277.08	302.97	6024.84	558.94	5535.03	289.72	9619.76	416.92	7928.71
GPT-5.4-mini	10	353.88	7696.76	302.97	5615.12	558.94	6324.61	255.85	10685.64	297.76	8161.68
GPT-5.4	6	386.70	6733.20	302.97	5041.82	558.94	4456.75	269.12	9736.57	415.78	7697.67
Combinatorial Optimization Solvers
DFS	4	392.01	6602.05	302.97	4187.31	558.94	4903.47	289.21	9619.76	416.92	7697.67
Online DP	2	392.14	5628.85	302.97	2573.66	558.94	2680.27	289.72	9619.76	416.92	7641.72
Offline DP	1	409.71	5554.29	302.97	2573.66	558.94	2680.27	289.72	9529.04	487.22	7434.19

SimWorld: grounded-topological planning

SimWorld reports Feasible (F), Completion (C), and Success Distance (SD). Higher F/C is better and lower SD is better.

Table 3. SimWorld scaling results under varying task counts.

Method	Modality	Average			n=5			n=8			n=10
Method	Modality	F	C	SD	F	C	SD	F	C	SD	F	C	SD
Qwen3-VL-8B	Multi.	0.832	0.823	743.6	0.877	0.917	477.7	0.698	0.686	829.7	0.921	0.867	923.3
Qwen3-VL-8B	Image	0.896	0.902	738.7	0.846	0.901	476.5	0.952	0.915	781.2	0.889	0.890	958.3
Qwen3-VL-8B	Text	0.649	0.125	683.7	0.873	0.222	447.4	0.661	0.097	710.0	0.413	0.056	893.7
Qwen3-VL-30B	Multi.	0.794	0.799	754.2	0.769	0.769	516.6	0.806	0.821	756.7	0.806	0.806	989.3
Qwen3-VL-30B	Image	0.429	0.780	931.9	0.477	0.822	535.3	0.349	0.752	1083.8	0.460	0.765	1176.5
Qwen3-VL-30B	Text	0.900	0.910	756.2	0.939	0.932	491.5	0.871	0.915	747.3	0.889	0.884	1029.8
Claude-Haiku-4.5	Multi.	0.499	0.908	760.0	0.846	0.932	451.0	0.397	0.939	988.6	0.254	0.852	840.3
Claude-Haiku-4.5	Image	0.126	0.672	785.6	0.139	0.699	354.1	0.143	0.667	879.5	0.095	0.649	1123.1
Claude-Haiku-4.5	Text	0.911	0.391	894.6	0.954	0.588	559.6	0.905	0.347	1594.9	0.873	0.238	529.3
Claude-Sonnet-4.6	Multi.	0.516	0.921	751.3	0.785	0.923	475.8	0.381	0.936	958.8	0.381	0.905	819.4
Claude-Sonnet-4.6	Image	0.125	0.674	654.8	0.200	0.702	328.8	0.111	0.677	697.2	0.064	0.644	938.4
Claude-Sonnet-4.6	Text	0.937	0.443	744.2	0.969	0.622	564.6	0.936	0.395	654.9	0.905	0.311	1013.1
GPT-5.4-mini	Multi.	0.462	0.914	794.5	0.846	0.914	412.2	0.286	0.925	745.0	0.254	0.902	1226.4
GPT-5.4-mini	Image	0.177	0.687	797.7	0.262	0.732	500.8	0.143	0.679	658.6	0.127	0.649	1233.8
GPT-5.4-mini	Text	0.921	0.383	510.6	0.892	0.551	541.4	0.936	0.323	509.7	0.936	0.276	480.8
GPT-5.4	Multi.	0.392	0.821	594.9	0.600	0.895	428.5	0.349	0.800	413.8	0.226	0.769	942.4
GPT-5.4	Image	0.313	0.785	584.7	0.400	0.828	291.3	0.318	0.782	526.0	0.222	0.744	936.7
GPT-5.4	Text	0.757	0.500	581.5	0.844	0.622	500.9	0.698	0.484	607.9	0.730	0.394	635.8

Table 4. SimWorld scene-wise results across five scene groups.

Method	Modality	Open			T1			T2			Indoor			Campus
Method	Modality	F	C	SD	F	C	SD	F	C	SD	F	C	SD	F	C	SD
Qwen3-VL-8B	Multi.	0.909	0.909	640.4	0.880	0.831	422.0	0.840	0.815	533.7	0.600	0.600	1705.8	0.733	1.000	2783.5
Qwen3-VL-8B	Image	0.909	0.943	646.9	0.933	0.929	411.2	0.907	0.932	516.1	0.533	0.572	1598.7	1.000	0.920	2851.3
Qwen3-VL-8B	Text	0.545	0.466	589.2	0.680	0.103	n/a	0.600	0.105	n/a	0.533	0.078	n/a	1.000	0.133	n/a
Qwen3-VL-30B	Multi.	1.000	1.000	645.5	0.867	0.878	414.0	0.693	0.693	554.3	0.600	0.600	1632.4	1.000	1.000	2738.6
Qwen3-VL-30B	Image	0.273	0.789	444.8	0.320	0.763	414.4	0.507	0.803	502.3	0.400	0.522	1514.4	0.733	1.000	2963.6
Qwen3-VL-30B	Text	0.909	0.909	671.2	0.960	0.973	415.9	0.880	0.894	517.4	0.600	0.600	1691.2	1.000	1.000	2830.4
Claude-Haiku-4.5	Multi.	0.818	0.973	549.2	0.533	0.916	329.0	0.413	0.937	444.1	0.400	0.587	1406.7	0.667	1.000	2192.2
Claude-Haiku-4.5	Image	0.182	0.645	381.0	0.107	0.685	407.1	0.120	0.637	429.3	0.200	0.482	1214.1	0.133	0.985	3158.0
Claude-Haiku-4.5	Text	0.909	0.725	374.6	0.920	0.342	268.5	0.947	0.337	437.0	0.600	0.338	1313.4	1.000	0.743	2059.7
Claude-Sonnet-4.6	Multi.	0.818	1.000	576.9	0.533	0.940	365.7	0.453	0.940	426.5	0.400	0.600	1334.3	0.667	1.000	2409.4
Claude-Sonnet-4.6	Image	0.091	0.673	406.5	0.160	0.683	319.9	0.107	0.639	418.7	0.200	0.512	1645.3	0.000	0.973	n/a
Claude-Sonnet-4.6	Text	1.000	0.761	393.7	0.960	0.385	279.9	0.973	0.406	403.2	0.600	0.405	1301.5	0.933	0.742	1922.9
GPT-5.4-mini	Multi.	0.636	0.959	599.7	0.507	0.936	330.6	0.387	0.948	398.0	0.467	0.537	1562.4	0.533	0.973	2345.6
GPT-5.4-mini	Image	0.182	0.652	437.3	0.133	0.675	356.1	0.187	0.687	413.5	0.200	0.473	1126.3	0.333	0.985	2368.5
GPT-5.4-mini	Text	0.636	0.598	414.1	0.973	0.377	252.5	0.960	0.323	419.7	0.600	0.298	992.1	1.000	0.668	1627.9
GPT-5.4	Multi.	0.545	0.861	385.1	0.307	0.834	255.8	0.467	0.839	325.1	0.267	0.490	1028.6	0.500	1.000	1810.8
GPT-5.4	Image	0.182	0.739	415.0	0.280	0.785	313.7	0.427	0.819	423.3	0.267	0.480	1641.6	0.067	0.953	3232.4
GPT-5.4	Text	0.909	0.848	425.8	0.827	0.504	193.1	0.720	0.446	206.7	0.333	0.375	857.2	0.929	0.637	1623.4

DualWorld-VQA: diagnostic examples and results

VQA items are organized into three layers. Perception-level questions test visual geometric relations; topology-level questions introduce wall constraints and valid walking routes; planning-level questions require grounded pickup-delivery decisions under capacity and precedence constraints.

Layer 1

Visual Perception

Example: order START, D1, D2, P1, P2 from left to right.

Layer 2

Spatial Topology

Example: decide whether path A->B or A->C is longer when walls block straight-line shortcuts.

Layer 3

Logical Planning

Example: find the visit order that minimizes total travel distance under capacity constraints.

Key Findings

The core bottleneck is alignment between perception and reasoning.

01

MLLMs still lag behind solvers.

Strong models recover much of the reward in TextWorld, but Offline DP remains higher and faster.

02

Online heterogeneous planning is brittle.

Mixed online release requires jointly handling priorities, deadlines, and repeated replanning.

03

Multimodal evidence can hurt feasibility.

Completion can rise in multimodal mode while executable feasibility falls sharply.

04

Topology changes what competence is tested.

Indoor scenes stress feasibility, campus scenes stress long-range efficiency, and T1/T2 separate completion from route quality.

05

VQA reveals a planning-level cliff.

Models handle isolated perception or topology better than the integrated planning decisions that require both.

Citation

Citation will be released after review.

@misc{dualworldbench_anonymous,
  title  = {DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?},
  author = {Anonymous},
  year   = {2026},
  note   = {Anonymous submission}
}