Constructing
3D Scenes with Code

Benchmarking coding agents that turn open-ended scene descriptions into engine-native 3D scenes in Unreal Engine. Measuring spatial reasoning, task fulfillment and physical validity.

Read the paper ↗Code & Data ↗Watch the teaser ↘

Code4Scene in 90 secondsConstruction and evaluation

Task Format

Nordic Harbour · Text-to-Scene

An open-ended scene description, an asset catalog and the Unreal Engine editor.

Agent inputText + assets

Task instruction

Build me a compact, sunlit Copenhagen-style canal district with narrow cobblestone streets and rectangular waterways enclosing dense urban blocks.

Townhouses, canals, bridges and a waterfront promenade — with detailed requirements for materials, layout and street furniture.

Given: an empty level and the content pack’s asset catalog.

Agent outputGPT-6 Astra

Loading 3D scene…Drag to orbit · Scroll to zoom

Compare agents and evaluator views ↗

EvaluationEngine-native scene

CASE SCORE0.877

Detailed Alignment: 0.757
Overview Alignment: 0.909
Physical Safety: 0.900

Task fulfillment, artifact integrity and static physical validity.

0.2 × Detailed + 0.6 × Overview + 0.2 × Physical

+ Full task instructionTEXT-TO-SCENE

Build me a compact, sunlit Copenhagen-style canal district with narrow cobblestone streets and rectangular waterways enclosing dense urban blocks. Arrange roughly thirty attached four- to six-storey townhouses in continuous street and canal-front rows, using slender façades, repetitive white-framed windows, arched doors and carriage entrances, modest cornices, and steep tiled roofs packed with dormers and brick chimneys. Include occasional exposed party walls between staggered building rows. Finish the façades in weathered muted red, mustard yellow, dusty blue, warm grey, beige, and exposed brick, with red, orange, and charcoal roof tiles. Run a principal waterfront street along a stone quay, crossed by smaller streets and linked across the canals by short, gently arched stone-and-metal bridges, including one opening onto a broad paved promenade with rounded landings. Form the canal edges from stepped pale stone, dark timber retaining walls, and iron railings around reflective rippling water. Add regularly spaced leafy trees, ornate black lamps, stone bollards, benches, European road signs, zebra crossings, puddles, grime, and sparse weeds.

20 scenes in the Research Track: affordable, reproducible evaluation for academic and model-development use160 scenes in the Full Benchmark: comprehensive evaluation for leaderboard and final reporting

Leaderboard

Research Track · 20 scenes · 19 coding-agent configurations · re-run in October 2026.

Text-to-scene generation with frontier models is a long-horizon agentic task with substantial inference cost and runtime, which makes repeated full-scale evaluation difficult for many research groups. The Research Track supports affordable, reproducible experimentation; the Full Benchmark provides the more comprehensive evaluation for final model comparison and leaderboard reporting.

Code4Scene Research TrackImage-to-Scene

20 scenes · score out of 100

Swipe to view every configuration →

Sorted by score. Bar labels are scores ×100, rounded down; the table truncates to three decimal places. One color per model provider.

Code4Scene Pareto Frontier

Quality against the median cost of one case · Research Track, 20 scenes

Swipe to explore all configurations →

Solid: the frontier — nothing cheaper scores higher. One point per configuration, at its evaluated reasoning effort. Cost is the median USD per case. Muse Wonder Full 1.0 (max) is sold by subscription; cost = the token counts its CLI logs, at Muse Wonder's per-token list price. Gemma 4 31B (thinking on) is self-hosted; cost = the token counts its agent logs, at Gemma 4 31B's OpenRouter list price ($0.20 input, $0.40 output per million).

Ranking

Research Track, 20 scenes, re-run in October 2026 with every agent in the same sandbox. Case score = 0.2 × Detailed + 0.6 × Overview + 0.2 × Physical; a case without a valid scene scores 0. Detailed, Overview and Physical are means of the per-case scores. Cost is the median per case. *Muse Wonder Full 1.0 (max) is sold by subscription; cost = the token counts its CLI logs, at Muse Wonder's per-token list price. *Gemma 4 31B (thinking on) is self-hosted; cost = the token counts its agent logs, at Gemma 4 31B's OpenRouter list price ($0.20 input, $0.40 output per million).

#	Agent configuration	Score	Detailed	Overview	Physical	Median cost
1	Claude Sonnet 5.5 (max)ParetoAnthropic	0.837	0.803	0.812	0.945	$26.93
2	Claude Opus 5.5 (max)ParetoAnthropic	0.832	0.763	0.818	0.943	$26.90
3	Claude Fable 5.1 (max)ParetoAnthropic	0.800	0.746	0.756	0.983	$23.42
4	GPT-6 Astra (max)ParetoOpenAI	0.790	0.760	0.811	0.759	$14.25
5	GPT-6.1 Sol (max)ParetoOpenAI	0.784	0.742	0.778	0.845	$2.59
6	Gemini 4 Argon (high)Google	0.777	0.688	0.756	0.930	$4.83
7	GPT-5.6 Sol (max)OpenAI	0.723	0.640	0.713	0.836	$6.87
8	Claude Haiku 5.5 (max)ParetoAnthropic	0.718	0.652	0.667	0.937	$2.43
9	Muse Wonder Full 1.0 (max)Meta	0.715	0.666	0.696	0.819	$3.66*
10	Gemini 3.8 Flash (high)Google	0.694	0.578	0.662	0.905	$3.85
11	GPT-6 Luna (max)ParetoOpenAI	0.688	0.606	0.666	0.834	$0.16
12	Grok 4.6 (xhigh)xAI	0.625	0.573	0.645	0.619	$7.41
13	Muse Spark Open (max)open weightsMeta	0.611	0.583	0.591	0.700	$2.01
14	Muse Spark 1.3 (max)Meta	0.548	0.517	0.547	0.582	$0.23
15	GLM-5.3 Flash (max)open weightsZ.ai	0.542	0.506	0.545	0.569	$0.94
16	Grok 4.7 (xhigh)xAI	0.541	0.521	0.546	0.549	$7.29
17	MiMo V2.6 Pro (max)open weightsXiaomi	0.527	0.484	0.507	0.630	$0.92
18	Gemma 4 31B (thinking on)open weightsParetoGoogle	0.517	0.448	0.469	0.731	$0.06*
19	DeepSeek V4.1 Flash (max)open weightsDeepSeek	0.366	0.343	0.321	0.526	$0.35

#	Agent configuration	Score	Repair F1	Physical	Indoor	Outdoor	Cost / case
1	Gemini 3.8 Flash (high)ParetoGoogle	0.581	0.527	0.796	0.717	0.513	$1.35
2	GPT-6 Astra (max)OpenAI	0.515	0.445	0.796	0.785	0.380	$3.32
3	Claude Opus 5 (max)Anthropic	0.468	0.389	0.786	0.618	0.393	$4.46
4	Claude Fable 5.1 (max)Anthropic	0.424	0.332	0.795	0.535	0.369	$3.09
5	GPT-5.6 Sol (high)OpenAI	0.393	0.319	0.689	0.497	0.341	$1.88
6	Muse Spark 1.3 (medium)ParetoMeta	0.358	0.246	0.807	0.633	0.221	$0.07
7	GLM-5.3 Flash (max)open weightsZ.ai	0.320	0.188	0.848	0.465	0.247	$0.09
8	Qwen 3.8 27B (thinking on)open weightsAlibaba	0.226	0.118	0.660	0.297	0.191	$0.26
9	DeepSeek V4.1 Flash (high)open weightsParetoDeepSeek	0.210	0.066	0.787	0.189	0.221	$0.05
10	Qwen 3.8 27B (thinking off)open weightsAlibaba	0.200	0.083	0.667	0.266	0.166	$0.28
11	Inkling (high)Thinking Machines	0.198	0.095	0.609	0.169	0.213	$0.75
12	Grok 4.6 (high)xAI	0.184	0.044	0.744	0.197	0.178	$0.26
13	Gemma 4 31B (thinking on)open weightsParetoGoogle	0.126	0.003	0.615	0.127	0.125	$0.03
14	Gemma 4 31B (thinking off)open weightsParetoGoogle	0.115	0.001	0.572	0.112	0.116	$0.02

Click a column to sort. Pareto: on the score-against-cost frontier.

Open the gallery: what each of the 19 configurations built on every scene, with its score and cost.

Cases

Visualizing agent outputs

Four agents build the same scene: a time-lapse of each build, then an orbit of the result. Pick a scene, or open the gallery for all 19 configurations on all 20 scenes.

Swipe to see all four agents →

Text-to-Scene · re-run, October 2026

Blue City

Build me a compact Moroccan medina street inspired by Chefchaouen, with a tranquil, sunlit atmosphere and roughly fifteen to twenty tightly adjoining two- to four-story buildings arranged along both sides of a broad, straight road. Layer the irregular façades with recessed courtyards, narrow alleys, horseshoe-arched passages, exterior stair landings, projecting timber galleries, small balconies, and overlapping rooflines. Construct the road from worn, slightly reflective paving with broad shallow puddles, pale raised curbs, and narrow dark cobbled sidewalks. Finish the buildings in weathered cobalt and powder blue, white, pale turquoise, dusty coral, and peach stucco, frequently exposing dark stone masonry beneath chipped plaster. Use predominantly orange terracotta roof tiles, punctuated by pale-blue tiled pyramidal roofs and flat blue terraces. Make a tall blue tower house with a dark wooden room projecting below its roof the central landmark. Frame deep-set wooden doors and arched windows with alternating blue-and-white bands, then add flowering vines, tall palms, twin-lantern streetlights, and occasional wooden benches.

Claude Opus 5.5 (max)0.747
GPT-6 Astra (max)0.814
GPT-6.1 Sol (max)0.834
Gemini 4 Argon (high)0.803

Compare all 19 agents on this scene ↗

Each tile: the build as a time-lapse, then an orbit of the scene the agent saved. Chips: the case score ×100, rounded down.

Loading 3D scene…Drag to orbit · Scroll to zoom

Text-to-Scene · Showcase

Industrial Harbor

GPT-6 Astra (max) · 3,949 actors, built in one agent session from an empty 130 m stage

An industrial harbor loading dock with shipping containers, barrels, cranes, warehouse props, pallets, and warning signs.

Open at full texture resolution ↗

Streams about 1.2 GB of textures on first load.

Open the gallery

Findings

Takeaways

What the paper's evaluation shows across its 14 agents. Every chart below is drawn from the paper's numbers, not the October 2026 Text-to-Scene re-run above; hover for values.

Takeaway

Spatial Composition remains the weakest requirement family for all 14 agents.

Errors persist even when the required objects are present: generating the right objects does not ensure that their relationships satisfy the specification.

Requirement-family score per agent

Identity & Environmentmean 67Content & Quantitymean 54Attributes & Materialsmean 48Spatial Composition, in the agent’s colourmean 36

One group per agent, in Text-to-Scene rank order: Identity & Environment, Content & Quantity, Attributes & Materials, then Spatial Composition, the lowest bar in every group. Scores ×100 (paper Figure 14b).

The objects are there; the relation is not

Loading 3D scene…Drag to orbit · Scroll to zoom

✓ slab roof 0.95✓ stone coping 0.82✗ slab roof enclosed by the coping 0.78

GPT-5.6 Sol’s saved Egyptian Temple scene. The judge finds the roof and stone coping, but flags their spatial relationship. View the judge’s evidence ↗

Mismatch when the objects are present

Share of checks that fail although every object they name is in the scene, across the benchmark (paper Figure 6a).

Citation

Cite This Work

@misc{ye2026code4scene,
  title         = {Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes},
  author        = {Xiaokang Ye and Siddhant Hitesh Mantri and Zimeng Chen and Edward Zhang and Zhaoxu Zheng and Yuanheng Li and Yizhao Chen and Tianyang Huang and Lianhui Qin},
  year          = {2026},
  eprint        = {2609.36777},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url           = {https://arxiv.org/abs/2609.36777},
}

arXiv GitHub Repository SimWorld Main Site

Constructing
3D Scenes with Code

Task Format

Nordic Harbour · Text-to-Scene

Old Industrial Pallet Bay · Image-to-Scene

Leaderboard

Code4Scene Research TrackImage-to-Scene

Code4Scene Pareto Frontier

Visualizing agent outputs

Blue City

Industrial Harbor

Old Industrial Pallet Bay

New York Mailbox Pair

Suburban Umbrella Pair

Rosie's Service Bin

Rosie's Roadside Billboard

Suburban Driveway Mailbox

Takeaways

Spatial Composition remains the weakest requirement family for all 14 agents.

Construction and editing probe different capabilities: scene-level spatial reasoning and precise control of scene state.

Editing ability varies substantially across repair types.

Recovering the target does not guarantee precise editing.

Cite This Work

Constructing3D Scenes with Code

Task Format

Nordic Harbour · Text-to-Scene

Old Industrial Pallet Bay · Image-to-Scene

Leaderboard

Code4Scene Research TrackImage-to-Scene

Code4Scene Pareto Frontier

Visualizing agent outputs

Blue City

Industrial Harbor

Old Industrial Pallet Bay

New York Mailbox Pair

Suburban Umbrella Pair

Rosie's Service Bin

Rosie's Roadside Billboard

Suburban Driveway Mailbox

Takeaways

Spatial Composition remains the weakest requirement family for all 14 agents.

Construction and editing probe different capabilities: scene-level spatial reasoning and precise control of scene state.

Editing ability varies substantially across repair types.

Recovering the target does not guarantee precise editing.

Cite This Work

Constructing
3D Scenes with Code