Overview
At a glance
SceneSmith builds dense indoor scenes from language using cooperating agents, generated and retrieved assets, and physical post-processing. Across 179 room and 31 house prompts, it produces many more objects than the evaluated baselines and wins most human comparisons. The demonstration of automatic robot evaluation is promising, but small and entirely simulated.
1 Introduction
The target is an interactable simulation environment: furniture, articulated assets and separate small objects with collision geometry and estimated dynamics. A visually plausible room alone does not supply these properties. The method combines structured construction with simulation checks, rather than training a single scene generator.
This edition covers all 69 pages of arXiv v2, dated 30 May 2026. The title and all five authors are verified. All figures, tables, four displayed equations, references and the 210 exact evaluation prompts are retained.
2 Related Work
The related work spans dataset- and program-driven scene synthesis, diffusion models, LLM layout systems and reconstruction of robot environments. HSM supplies hierarchical scene motifs and a support-surface detector; Holodeck supplies an important language-conditioned room/house baseline. SceneSmith adds on-demand rigid assets, explicit manipulands and simulation preparation, with coverage and cost constraints that remain distinct from visual realism.
3 Agentic Scene Construction for Simulation-Ready Environments
Five construction stages progressively add layout, furniture, wall objects, ceiling objects and manipulands. Agents inspect rendered and structured state, use geometry-aware tools, and revise placements. Physics post-processing then repairs some remaining intersections and unstable configurations.
3.1 Scene Hierarchy
Room branches run independently after global layout. Within a room, furniture precedes wall and ceiling objects; each selected supporting entity then gets its own manipuland agent. Most placements use a surface-local position and yaw, lifted into a full pose. Composite tools let physics produce less restrictive three-dimensional arrangements.
3.2 Agentic Trio: Designer, Critic, and Orchestrator
A designer changes the scene; a critic returns category scores and explanations without modifying it; an orchestrator requests revisions, manages checkpoints and decides when to stop. The memory keeps recent full turns and summarizes older ones, while image context is bounded. A rollback restores scene state but preserves the history of the failed attempt.
3.3 Asset Generation and Routing
An asset router rejects requests belonging to another stage, splits composites into independently manipulable pieces, and selects generation, articulated retrieval or flat texture generation. Image-to-mesh assets receive collision geometry, canonical axes, estimated mass/friction and validation. Articulation comes from a finite library; generated fallback objects are static.
3.4 Simulation Readiness
Local feasibility tools guide placement, then translation-only nonpenetration optimization and gravity settling prepare the scene in Drake. These steps are applied after furniture and manipuland construction. Wall/ceiling fixtures stay welded; some unstable objects are removed. Thus “all scenes used as generated” permits internal retries, repair and deletion.
3.5 Application: Robot Policy Evaluation
A task prompt is expanded into varied scene descriptions with necessary objects and initial conditions. A robot acts in each scene, and an evaluator checks geometric relations, consulting rendered views for ambiguous cases. This creates task-conditioned tests on demand; it does not establish that generated scenes are free of all training-data overlap.
4 Evaluation
Evaluate prompt adherence, human visual preference, physical checks and an illustrative robot-policy comparison. Room and house protocols differ: all external systems enter the 179-room comparison, but only Holodeck supports the 31-house comparison. More objects and higher user preference are not themselves proofs of downstream policy transfer.
4.1 Experimental Setup
The corpus combines SceneEval prompts with room-type, high-density, themed and multi-room extensions. Five external systems yield six configurations because LayoutVLM uses two asset libraries. Six internal ablations remove selected capabilities. Baselines retain released models and prompts; additional GPT-5.2 reruns of HSM/Holodeck partially control the base-model difference.
4.2 Scene Generation Results
Room scenes average 71.1 objects versus 11.2–23.0 for external baselines. Averaged external comparisons favor SceneSmith 92.2% for realism and 91.5% for faithfulness; house wins are 80.3% and 84.7%. Room collision rate is 1.2% and static-stability rate 95.6%, while navigation/accessibility can be worse in denser scenes.
Generated assets and asset validation have clear user-study effects. Removing criticism, specialized tools or memory produces nonsignificant human differences under this study’s power; that does not establish equivalence. Removing criticism lowers mean cost from $13.98 to $4.24 and mean object count from 71.1 to 54.0.
Original selectable source
| Realism | Faithfulness | ||||||||
| Comparison | #Obj | Win% | 95% CI | Sig. | Win% | 95% CI | Sig. | ||
| External Baselines (SceneSmith averages objects) | |||||||||
| vs HSM | 88.5 | [83.5, 92.1] | 0.88 | ** | 85.2 | [79.7, 89.3] | 0.78 | ** | |
| vs Holodeck | 88.6 | [83.8, 92.2] | 0.88 | ** | 90.6 | [85.9, 93.8] | 0.95 | ** | |
| vs SceneWeaver | 91.7 | [87.3, 94.7] | 0.99 | ** | 92.9 | [88.6, 95.6] | 1.03 | ** | |
| vs I-Design | 94.1 | [90.1, 96.5] | 1.08 | ** | 90.6 | [85.9, 93.8] | 0.95 | ** | |
| vs LayoutVLM (Curated) | 94.9 | [91.1, 97.1] | 1.12 | ** | 95.8 | [92.3, 97.8] | 1.16 | ** | |
| vs LayoutVLM (Objaverse) | 95.4 | [91.7, 97.5] | 1.14 | ** | 93.9 | [89.9, 96.4] | 1.07 | ** | |
| Ablations | |||||||||
| vs NotGenerated | 63.8 | [57.2, 69.9] | 0.28 | ** | 67.0 | [60.0, 73.3] | 0.35 | ** | |
| vs NoAssetValidation | 63.0 | [56.3, 69.1] | 0.26 | ** | 62.2 | [54.8, 69.1] | 0.25 | * | |
| vs NoObserveScene | 61.5 | [54.9, 67.7] | 0.23 | * | 53.2 | [45.7, 60.5] | 0.06 | – | |
| vs NoSpecializedTools | 54.8 | [48.2, 61.2] | 0.10 | – | 53.2 | [45.7, 60.5] | 0.06 | – | |
| vs NoAgentMemory | 53.4 | [46.8, 59.9] | 0.07 | – | 55.1 | [47.5, 62.4] | 0.10 | – | |
| vs NoCritic | 51.8 | [45.2, 58.4] | 0.04 | – | 47.5 | [40.4, 54.8] | 0.05 | – | |
| House-Level (SceneSmith averages objects) | |||||||||
| vs Holodeck | 80.3 | [63.5, 91.0] | 0.65 | ** | 84.7 | [67.5, 93.8] | 0.76 | ** | |
Original selectable source
| Method | #Obj | CNT | ATR | OOR | OAR | ACC | NAV | COL | STB | OOB |
|---|---|---|---|---|---|---|---|---|---|---|
| HSM | 22.72.6 | 60.64.2 | 61.57.3 | 30.65.7 | 66.26.6 | 88.91.7 | 99.40.4 | 20.63.0 | 45.23.9 | 5.51.1 |
| Holodeck | 23.01.4 | 44.24.3 | 44.47.2 | 16.84.6 | 38.07.1 | 84.91.4 | 99.60.2 | 12.32.3 | 31.92.7 | 0.80.4 |
| SceneWeaver | 13.51.0 | 41.83.9 | 29.16.7 | 15.34.2 | 31.66.5 | 77.52.7 | 98.11.2 | 12.52.5 | 37.34.4 | 0.00.0 |
| I-Design | 13.01.0 | 69.34.0 | 50.37.2 | 28.65.5 | 66.66.7 | 70.12.8 | 95.91.4 | 3.01.5 | 60.84.6 | 4.32.0 |
| LayoutVLM (Cur.) | 11.21.6 | 41.04.0 | 25.66.6 | 14.14.0 | 22.46.1 | 93.81.9 | 99.70.2 | 25.94.4 | 19.43.6 | 6.22.1 |
| LayoutVLM (Obj.) | 14.21.7 | 55.54.7 | 34.37.0 | 20.75.0 | 17.45.6 | 91.51.6 | 98.70.8 | 28.93.5 | 8.11.7 | 5.11.4 |
| SceneSmith (Ours) | 71.113.0 | 82.93.4 | 74.46.2 | 67.65.8 | 80.65.5 | 83.41.5 | 97.61.1 | 1.20.6 | 95.61.7 | 0.20.4 |
| House-Level | ||||||||||
| Holodeck | 81.318.3 | 64.66.8 | 43.227.6 | 30.110.1 | 58.46.3 | 79.43.0 | 99.60.2 | 3.83.1 | 17.95.6 | 0.30.2 |
| SceneSmith (Ours) | 214.160.9 | 83.36.8 | 66.824.6 | 76.69.2 | 82.65.8 | 79.92.0 | 96.52.1 | 0.90.5 | 79.89.1 | 0.60.3 |
4.3 Robot Policy Evaluation
A model-based mobile iiwa policy succeeds on 16 of 100 generated task scenes; a deliberately degraded version succeeds on 12. The evaluator agrees with human labels on 299 of 300 tested states. No confidence interval or significance test is supplied for the four-success policy gap, so it is a demonstration of the evaluation workflow rather than a statistically established separation.
4.4 Robot Simulation Demonstrations
Figure 5 shows teleoperated RB-Y1 manipulation and a prior language-conditioned visuomotor policy moving an apple from a bowl to a cutting board without SceneSmith-specific training. These are qualitative simulation examples, not a quantitative generalization benchmark or a real-robot transfer study.
5 Conclusion
The strongest evidence concerns dense, prompt-conditioned scenes that work directly in the chosen simulator and are preferred to the tested scene generators. Generated materials, articulated coverage, estimated physical parameters, sequential construction and substantial generation cost remain limitations. Broad claims that scene creation is no longer a robotics bottleneck are aspirational.
Acknowledgements
The authors acknowledge discussions with Hongkai Dai, Jeremy Binagia, Peter Werner and Tom Erez, and Patrick Farrell’s video assistance. Funding includes Amazon, the Office of Naval Research, Toyota Research Institute and an NSF graduate fellowship; the paper states the authors’ views rather than those of the funders.
Impact Statement
The authors position the work as enabling scalable robot simulation and evaluation, with no specific additional societal consequences singled out in their impact statement.
Appendix
The appendix supplies implementation details, complete evaluation protocols, all 210 prompts, interface screenshots, qualitative results and explicit failure cases.
| A. System Architecture | A |
| B. Asset Acquisition | B |
| C. Layout Generation | C |
| D. Furniture Placement | D |
| E. Wall Object Placement | E |
| F. Ceiling Object Placement | F |
| G. Manipuland Placement | G |
| H. Physical Feasibility Post-Processing | H |
| I. Stochastic Placement | I |
| J. Export and Simulator Compatibility | J |
| K. Comparison to Prior Scene Generation Systems | K |
| L. Task-Driven Robot Evaluation | L |
| M. Evaluation Methodology | M |
| N. Human Evaluation | N |
| O. Automated Evaluation | O |
| P. Evaluation Prompts | P |
| Q. User Study Interface Screenshots | Q |
| R. Additional Qualitative Results | R |
| S. Limitations and Failure Analysis | S |
Appendix A System Architecture
All five stages share a designer–critic–orchestrator loop while exposing tools appropriate to their geometric task. The hierarchy limits context size and enables independent room and support-entity processing.
A.1 Agent Roles
Designers place or modify objects and observe state. Critics score categories from 0 to 10 and describe defects. Orchestrators invoke initial design, refinement and critique, reset checkpoints, and choose natural or precise placement style. Figure 6 shows revisions and a rollback after a worse pharmacy layout.
| Tool | Description |
|---|---|
| request_initial_design | Invoke designer for initial scene |
| request_design_change | Invoke designer for refinement |
| request_critique | Invoke critic for evaluation |
| reset_scene_to_checkpoint | Revert to previous checkpoint |
| select_placement_style | Set natural/perfect placement mode |
A.2 Model Configuration
The reported configuration uses GPT-5.2 for agents and VLM calls. Designers and critics use high reasoning effort; orchestrators use low effort; auxiliary routing, physics estimation and validation use medium effort. These are the paper’s experimental settings, not a comparison establishing optimal model choices.
A.3 Scene Observation
Structured queries return object IDs, support-surface poses, bounding boxes, sizes and descriptions, or room/wall topology. Visual observations add stage-specific labels, grids and viewpoints. The two channels complement each other: precise numeric geometry does not replace inspection of shape and semantic appearance.
A.4 Checkpoint and Rollback
After critique, retain both the current and previous complete scene checkpoints with scores. If quality falls beyond thresholds, restore the previous scene and ask for another approach. Agent context remains, so the restored geometry does not erase evidence about unsuccessful modifications.
A.5 Parallel Scene Generation
A shared geometry service distributes text-to-3D jobs across one worker per GPU. Batched asynchronous requests overlap generation with CPU mesh and physical-property processing. Reported throughput uses 25 concurrent scenes on eight NVIDIA L40S GPUs; it should not be read as a measured optimized single-scene deployment latency.
Appendix B Asset Acquisition
Asset acquisition separates shape creation or retrieval from collision, physical-property and quality processing. Each placed small object remains an independent simulation object even when a composite placement tool arranges it with others.
B.1 Asset Router
Routing first filters stage-inappropriate requests, then decomposes composite descriptions, then chooses generated rigid geometry, retrieved articulation or a thin covering. For example, a bowl and fruit become separate assets, while a desk and monitor belong to different construction stages. This classification can fail, as the fused TV example later shows.
B.2 Text-to-3D Generation
Generate a 1024-square reference image with GPT Image 1.5 on an opaque background, segment the object with SAM3, and reconstruct its surface with SAM3D. This is a single-image reconstruction pipeline, not direct recovery of measured shape or materials.
B.3 Articulated Object Retrieval
Moving-part objects come from ArtVIP; PartNet-Mobility was tested but judged too small or insufficient for the desired articulation. Average eight CLIP ViT-H-14 view embeddings into a 1,024-dimensional descriptor. Filter object type, retrieve top semantic matches, then optionally rerank by L1 bounding-box-size error.
B.4 Thin Covering Generation
Rugs, posters and other flat surfaces use rectangular or circular geometry with tileable or single-image textures. Retrieve ambientCG materials by CLIP similarity, otherwise generate an image and add flat normals/uniform roughness; edge blending reduces tiling seams. Floor and manipuland coverings are visual-only and lack collision geometry.
B.5 Physical Property Estimation
A VLM estimates physical attributes from rendered asset views. The resulting parameters make simulation possible, but plausible estimates are not experimentally identified real-world dynamics.
B.5.1 Rigid Body Estimation
Six views provide one of 19 material classes, mass with a plausible range, and canonical up/front axes. Material class supplies friction; uniform density supplies inertia through Equation 1. Meshes become Z-up/Y-forward, with placement-dependent origin conventions. A mass range supports randomization without guaranteeing calibrated uncertainty.
Equation 1
| (1) |
B.5.2 Articulated Body Estimation
For articulated assets, first estimate total mass, then assign link materials and masses constrained to sum to that total. Render the complete object and isolated links with their sizes. Compute each link’s inertia from its geometry and assigned mass; inherited joint structure is preserved.
B.6 Collision Geometry
V-HACD decomposes visual meshes into collision hulls: up to 128 for furniture, 64 for wall objects/manipulands and 16 for ceiling fixtures. The authors found CoACD slower and more inflated for these assets. Their reported one-to-two-orders-of-magnitude speed difference is an observation in this pipeline, not a universal decomposition benchmark.
B.7 Asset Validation
Validation checks requested type/style, single-object structure, completeness, proportions and closed static drawers/doors. Text, metallic appearance and transparency receive leniency; retrieved articulated assets also get looser cosmetic checks. Failed assets are retried within a budget, then feedback returns to the designer for another asset or placement strategy.
Appendix C Layout Generation
The layout agent jointly manages room dimensions, adjacency, openings and materials. It propagates room-specific prompts to later stages. Tools may be called in varying order, with critique driving regeneration or local architectural changes.
| Tool | Category | Access | Description |
|---|---|---|---|
| observe_scene | Visual | D, C | Top-down render with room labels |
| render_ascii | Visual | D, C | ASCII floor plan with wall labels |
| generate_room_specs | Modification | D | Create rooms with dimensions |
| resize_room | Modification | D | Adjust room dimensions |
| add_adjacency | Modification | D | Add room connection |
| remove_adjacency | Modification | D | Remove room connection |
| add_open_connection | Modification | D | Create open floor plan |
| remove_open_connection | Modification | D | Remove opening |
| set_wall_height | Modification | D | Set wall height |
| add_door / remove_door | Modification | D | Manage doors |
| add_window / remove_window | Modification | D | Manage windows |
| get_material | Asset | D | Semantic material search |
| set_room_materials | Modification | D | Assign floor/wall materials |
| validate | Feasibility | D, C | Check placement validity |
C.1 Room Layout Creation
Extract room-specific items, global style and constraints from the scene prompt. Resolve ambiguous object locations to suitable rooms and carry quantity/negative constraints downstream. These derived prompts guide furniture, wall, ceiling and manipuland agents; incorrect early assignment can affect every later stage.
C.2 Room Placement Algorithm
Place rooms through best-first backtracking over a fixed room order and discrete attachment candidates. Complete layouts receive a global score; a timeout returns the best found. The stated completeness/optimality applies only to that fixed ordering and discretized position space when search completes, not unrestricted continuous floor-plan optimization.
C.2.1 Room Ordering
Rooms with no adjacency requirement act as anchors and are placed first. Connector rooms are scheduled once a required neighbor exists. Within these groups, larger rooms precede smaller ones to handle stronger geometric constraints early; changing this order can change the reachable result.
C.2.2 Edge-Based Attachment
Attach each new room to edges of placed rooms at 11 sampled edge positions. Nonsquare rooms also try a 90-degree rotation. The discrete candidate set makes search tractable while excluding many otherwise possible continuous arrangements.
C.2.3 Placement Scoring
Local scoring rewards adjacency to required existing neighbors and penalizes distance to the current layout centroid; candidates missing required adjacencies are rejected. Global scoring rewards occupied-area/bounding-box compactness and retention of previous room centers. Equation 3 uses the unsquared Euclidean displacement in its exponential, as printed.
Equations 2–3
| (2) |
Original selectable MathML
| (3) |
C.3 Architectural Elements
Add wall height, doors, windows, open connections and materials after room placement. Doors/windows choose a wall segment, left/center/right region and dimensions, then sample a position within that region; windows also set sill height. Open connections remove the shared wall, and editing can resize or regenerate the layout.
C.4 Layout Validation
Validation checks successful placement of every room and graph connectivity through doors/open connections from exterior doors. It requires at least one exterior door and a path to every room. This architectural graph check differs from later robot-footprint collision and reachability checks.
C.5 Critic Evaluation
The layout critic scores room proportions, spatial flow, natural light, material consistency and prompt following. These model judgments guide revisions; they do not replace the explicit connectivity check.
C.6 Observation
Figure 8 preserves both rendered layout/material swatches and the ASCII floor plan with wall labels, dimensions, doors and windows. The text view supplies exact segment references for subsequent architectural edits.
Appendix D Furniture Placement
Each room receives a fresh furniture agent after its architecture is fixed. Placement uses floor-centered x/y and yaw; z, roll and pitch are fixed for supported furniture. Table 5 preserves the state, asset, modification, orientation, snapping, physics and reachability tool interface.
| Tool | Category | Access | Description |
|---|---|---|---|
| observe_scene | Visual | D, C | Multi-view renders with annotations |
| get_current_scene_state | State | D, C | Object positions and bounding boxes |
| generate_assets | Asset | D | Create 3D furniture from text |
| list_available_assets | State | D, C | List generated assets |
| add_furniture | Modification | D | Place at floor position |
| move_furniture | Modification | D | Reposition existing furniture |
| remove_furniture | Modification | D | Remove from scene |
| rescale_furniture | Modification | D | Uniform scaling |
| check_facing | Specialized | D, C | Verify orientation relationships |
| snap_to_object | Specialized | D | Align and eliminate gaps |
| check_physics | Feasibility | D, C | Detect collisions |
| check_reachability | Feasibility | D, C | Verify traversability |
D.1 check_facing Algorithm
check_facing uses an object’s canonical forward axis and a ray–bounding-box test against another object. It chooses the target center for a volume-ratio “circular” heuristic and otherwise the closest box point, then returns an orientation correction. This heuristic addresses semantic facing without relying solely on a camera view.
D.2 snap_to_object Algorithm
snap_to_object first separates overlapping boxes, then optionally rotates toward/away from the target, then advances in small steps until the next collision and retains the last collision-free pose. Its expanded square source box guards against rotation reintroducing overlap. This is a staged local adjustment, not a globally optimal placement solver.
D.3 check_reachability Algorithm
Model the robot as a configurable disk: erode the floor, dilate oriented furniture boxes, subtract obstacles, then test connectivity of the remaining free region. Report its largest-component fraction and objects whose removal improves connectivity. This is a two-dimensional footprint proxy, not whole-body or dynamically feasible navigation.
D.4 check_physics Tool
Checks report signed-distance furniture/wall intersections, rug overlaps and boundary violations, blocked doors/open connections and window-clearance warnings. Rugs use planar boxes because they have no collision mesh. Window blocking is a warning for agent judgment; a low desk can be acceptable where a tall bookshelf is not.
D.5 Critic Evaluation
Six critic categories cover realism, functionality, layout, holistic completeness, prompt following and reachability. Completeness can add typical furnishings absent from a literal prompt, whereas prompt following checks explicitly requested items and quantities. The distinction helps explain higher object counts.
D.6 Observation
Observe one annotated top-down view plus four corner views with obstructing walls hidden. Grids, bounding boxes, unique object labels, forward arrows and opening labels support spatial edits. Context and side views help resolve ambiguities that top-down boxes alone cannot.
Appendix E Wall Object Placement
One agent places all wall objects in a room. Each wall has local horizontal/vertical coordinates and rotation around its normal; objects sit flush at zero normal offset. Precomputed allowable wall regions exclude doors, windows and open connections.
| Tool | Category | Access | Description |
|---|---|---|---|
| observe_scene | Visual | D, C | Multi-view renders with annotations |
| get_current_scene_state | State | D, C | Wall surfaces and placed objects |
| list_wall_surfaces | State | D, C | Walls with excluded regions |
| generate_wall_assets | Asset | D | Create wall-mounted assets |
| list_available_assets | State | D, C | List generated assets |
| place_wall_object | Modification | D | Place using wall-local SE(2) |
| move_wall_object | Modification | D | Reposition (can change walls) |
| remove_wall_object | Modification | D | Remove from scene |
| rescale_wall_object | Modification | D | Uniform scaling |
| check_physics | Feasibility | D, C | Detect collisions |
E.1 Physics Validation
Physics checks flag wall-object collisions, room/ceiling boundary violations and intersections with excluded opening regions. Their output identifies objects so the designer can make targeted corrections.
E.2 Critic Evaluation
The wall critic evaluates realism, functionality, layout, holistic decoration and prompt following. It considers realistic mounting height and whether objects remain accessible around furniture.
E.3 Observation
A room overview gives context; an orthographic view of every wall adds wall-local grids, surface IDs, openings and labels. Nearby furniture is shown without extra labels to reveal interference while limiting clutter. Figure 10 retains all five views.
Appendix F Ceiling Object Placement
Ceiling fixtures are placed after furniture and wall objects, using room-local x/y/yaw at fixed ceiling height. Each room is processed separately, with the tool set in Table 7.
| Tool | Category | Access | Description |
|---|---|---|---|
| observe_scene | Visual | D, C | Multi-view renders with annotations |
| get_current_scene_state | State | D, C | Room bounds and ceiling objects |
| generate_ceiling_assets | Asset | D | Create ceiling-mounted assets |
| list_available_assets | State | D, C | List generated assets |
| place_ceiling_object | Modification | D | Place using room-local SE(2) |
| move_ceiling_object | Modification | D | Reposition fixture |
| remove_ceiling_object | Modification | D | Remove from scene |
| rescale_ceiling_object | Modification | D | Uniform scaling |
| check_physics | Feasibility | D, C | Detect collisions |
F.1 Physics Validation
Check ceiling-fixture collisions with each other and with the rest of the scene. Their later fixed attachment means gravity settling will not repair a wrong ceiling pose.
F.2 Critic Evaluation
Four categories assess realistic fixture locations, lighting functionality, arrangement/clearance and requested items. Unlike furniture and wall critique, there is no separate holistic-completeness category here.
F.3 Observation
Observe one ceiling-level annotated top view and two side views. Unlabeled furniture/wall context helps reveal clearance and height problems; fixture IDs and coordinates support edits.
Appendix G Manipuland Placement
Select supporting entities, then run one agent per entity across all its surfaces. Local x/y/yaw placement references a surface ID, with surface boundaries checked automatically. Furniture, wall objects and architectural elements can serve as supports; composite tools produce stacks, filled containers, arrangements and piles.
| Tool | Category | Access | Description |
|---|---|---|---|
| observe_scene | Visual | D, C | Multi-view renders with annotations |
| get_current_scene_state | State | D, C | Surface bounds and placed objects |
| list_support_surfaces | State | D, C | Surfaces with area and clearance |
| generate_manipuland_assets | Asset | D | Create small object assets |
| list_available_assets | State | D, C | List generated assets |
| place_manipuland | Modification | D | Place using surface-local SE(2) |
| move_manipuland | Modification | D | Reposition object |
| remove_manipuland | Modification | D | Remove from scene |
| rescale_manipuland | Modification | D | Uniform scaling |
| create_stack | Specialized | D | Vertical stacking |
| fill_container | Specialized | D | Fill cavity with objects |
| create_arrangement | Specialized | D | Controlled placement on container |
| create_pile | Specialized | D | Random scattered arrangement |
| check_physics | Feasibility | D, C | Detect collisions |
G.1 Entity Selection and Prompt Derivation
A VLM selects entities to populate and supplies required versus optional objects, quantity/negative constraints and style notes. Nearby context such as chairs helps orient table settings. Missing a support entity can move intended objects to an incorrect fallback surface.
G.2 Support Surface Detection
Use HSM’s mesh-based horizontal support detector. Articulated link geometry is merged at zero joint positions, with drawers/doors closed, before clearance computation. Rendering opened articulation later aids observation but does not change this support-extraction configuration.
G.3 Physics Validation
Check manipuland–manipuland and manipuland–furniture collisions relevant to the current entity. Context filtering excludes violations involving objects the current agent cannot address. This improves actionable feedback but leaves whole-scene reconciliation to later processing.
G.4 Critic Evaluation
Critique covers realistic use, functionality, spatial arrangement, surface completeness and prompt requirements. Adding plausible optional clutter can improve completeness while also making traversal or manipulation harder.
G.5 Observation
Show a top view for each surface and four side views. Open articulated doors; for stacked drawers, show one open drawer at a time to avoid occlusion. Figures 12–14 retain the single-surface, multi-surface and articulated observation examples.
G.6 Composite Object Tools
Composite tools place multiple individual assets together and use gravity to determine final relative transforms. The composite is positioned on a support surface, but its children become independent simulation objects with full poses. Each tool has a different partial-success/failure rule.
G.6.1 create_stack
create_stack starts bottom-to-top using collision-box heights, then lets objects settle. If any item falls or exceeds the displacement threshold, reject the entire stack; also reject excessive height for available clearance. Nesting and lateral shifts can arise naturally during settling.
G.6.2 fill_container
fill_container estimates an inward-shrunk convex opening from top-rim vertices. Orient elongated items upright, sample horizontal positions and stagger heights before settling. Classify containment by final height; respawn fallen objects iteratively, which can dislodge previously successful ones. Success requires only one object to remain, and unsettled leftovers are removed.
G.6.3 create_arrangement
create_arrangement accepts specified local x/y/yaw on a circular or rectangular container. Only item centers must lie within its boundary; overhanging geometry is allowed. Reject pairwise initial collisions, then settle and reject the entire arrangement if any item falls. The agent must explicitly retry with changed positions.
G.6.4 create_pile
create_pile samples positions uniformly over a disk and orientations uniformly in SO(3), with vertically staggered spawns. After settling, retain items passing the height test and require at least two survivors; no internal retry occurs. Unlike fill_container, it can populate built-in sink cavities without a separate container asset.
Appendix H Physical Feasibility Post-Processing
Post-processing repairs residual penetrations and unstable placements after agent construction. Wall/ceiling fixtures stay welded, and decorative floor/manipuland coverings are visual-only. Physical readiness is measured for this modeled system, not validated against real material behavior.
H.1 Non-Penetration Projection
Following prior physics-aware scene projection, minimize squared displacement from initial translations subject to pairwise signed distance at least 10⁻⁵ m. Orientations remain fixed. Drake inverse kinematics and SNOPT solve the nonconvex problem; “nearest collision-free scene” is an objective, not a certified global projection.
Equations 4–4
Original selectable MathML
| (4) |
H.2 Gravity Settling
Run gravity simulation after nonpenetration projection to settle objects into supported configurations. The procedure can change intended poses and does not guarantee that every generated object remains in the final scene.
H.3 Pipeline Integration
Repair/settle furniture with furniture free to move; repair each support entity’s manipulands with that entity welded; finally repair all manipulands with all furniture welded. These three scopes keep local processing manageable but prevent later manipulands from moving furniture to create space.
H.4 Fallen Object Removal
Remove tipped furniture using its tilt from vertical. Remove manipulands that penetrate the floor or move down from their intended support toward floor level. These deletion rules help final physical metrics while potentially changing requested counts or semantic placement.
Appendix I Stochastic Placement
Gaussian position/orientation noise breaks the agents’ tendency toward round numbers and perfectly aligned layouts. The perturbation depends on object class and requested style; Table 9 preserves every standard deviation.
I.1 Mode Selection
Natural is the default and is also selected by words such as lived-in, cozy or casual. Pristine/showroom/gallery selects perfect. Natural standard deviations include 3 cm/1° for furniture and 1 cm/3° for manipulands; perfect uses 1 mm/0.1° across classes, with wall-height noise specified separately.
| Object Type | Parameter | Natural | Perfect |
|---|---|---|---|
| Furniture | Position XY | 3 cm | 1 mm |
| Furniture | Yaw | 1.0° | 0.1° |
| Wall | Position X (along wall) | 2 cm | 1 mm |
| Wall | Position Z (height) | 1 cm | 1 mm |
| Wall | Rotation | 0.5° | 0.1° |
| Ceiling | Position XY | 2 cm | 1 mm |
| Ceiling | Yaw | 0.5° | 0.1° |
| Manipuland | Position XY | 1 cm | 1 mm |
| Manipuland | Yaw | 3.0° | 0.1° |
I.2 Style-Aware Evaluation
Critics receive the placement style: slight misalignment can be acceptable for natural scenes, while perfect scenes demand tighter alignment. This changes judgment criteria as well as injected noise.
Appendix J Export and Simulator Compatibility
Native construction and checks use Drake SDF models and directives. MJCF/USD export carries geometry, collision models, physical parameters and joints for other supported simulators; Blender export carries visuals only. Format compatibility does not demonstrate numerically equivalent dynamics across engines.
Appendix K Comparison to Prior Scene Generation Systems
Table 10 contrasts stated capabilities with Holodeck, HSM, SceneWeaver, I-Design, LayoutVLM and ProcTHOR. “Novel assets” means generated shapes rather than selecting from a fixed library; separable composites remain independently manipulable. This feature checklist includes ProcTHOR although it is not an experimental baseline.
Original selectable source
| Feature | Ours | Holodeck | HSM | SceneWeaver | I-Design | LayoutVLM | ProcTHOR |
|---|---|---|---|---|---|---|---|
| Novel static asset generation | |||||||
| Language-conditioned generation | |||||||
| Multi-room scenes | |||||||
| Floor/wall materials and textures | |||||||
| Separable multi-object compositions | |||||||
| Articulated objects | |||||||
| Estimated physical properties | |||||||
| Multiple simulator exports | |||||||
| Iterative critique/refinement |
Appendix L Robot Policy Evaluation
The policy-evaluation pipeline generates task-conditioned scenes, runs a policy and checks resulting states. Figure 19 shows this loop; the following sections separate scene generation, evaluator design and the illustrative controller.
L.1 Scene Prompt Generation
Extract room requirements, necessary objects and initial-state constraints, while varying styles, variants and layouts. Crucially, avoid starting already at the goal: fruit must not initially be on the destination plate. Multi-room tasks can request several connected spaces.
L.2 Evaluator Agent
Enumerate candidate object pairs for the task, checking distance, support, contact and overlap through structured tools. Consult top-down/side renders for ambiguous containment or relationships. The evaluator reuses agent infrastructure and avoids one hand-coded predicate per task, but its visual decisions can still be wrong.
| Tool | Category | Description |
|---|---|---|
| list_objects | State | List all objects with positions |
| get_object_info | State | Object dimensions and orientation |
| get_distance | State | Signed distance between object pair |
| get_spatial_relation | State | Vertical/horizontal gaps, overlap |
| get_support | State | Contact detection and surface overlap |
| observe_scene | Vision | Multi-view renders of entire scene |
| observe_objects | Vision | Focused renders of specific objects |
L.3 Example Policy
The mobile iiwa policy parses language into predicates/object categories, ranks bindings, and uses segmented simulator point clouds for antipodal grasps. Collision-aware IK finds pick/place and approach poses; RRT-Connect, shortcutting and time-optimal parameterization produce motions. Extra clearance and gripper velocity/acceleration limits reduce drops. The degraded policy removes those two protections; this is not an end-to-end RGB generalist-policy comparison.
L.4 Experimental Results
Each of four tasks has 25 scenes. Standard/degraded successes are 3/3 for fruit-to-plate, 6/4 for can-to-table, 4/3 for cup-to-sink and 3/2 for bottle-to-coffee-table: 16/100 versus 12/100 overall. No uncertainty analysis is reported.
The evaluator’s 299/300 agreement comes from 100 scenes each inspected initially and after both policies. The sole disagreement is fruit on a plate edge. These related states validate this protocol, not unrestricted accuracy on independent tasks or scenes.
| Task | Standard | Degraded | |
|---|---|---|---|
| Fruit plate | 3/25 (12%) | 3/25 (12%) | 0% |
| Coke can table | 6/25 (24%) | 4/25 (16%) | +8% |
| Cup sink | 4/25 (16%) | 3/25 (12%) | +4% |
| Water bottle coffee table | 3/25 (12%) | 2/25 (8%) | +4% |
| Overall | 16/100 (16%) | 12/100 (12%) | +4% |
Appendix M Evaluation Methodology
Methodological details define what the reported semantic and physical scores measure, how baselines become simulatable and which prompts enter each comparison.
M.1 Experimental Setup
External baselines use open-source releases; internal ablations preserve the main pipeline while removing specific components. Room and house results are reported separately to respect baseline capability differences.
M.1.1 Baselines
HSM builds retrieved scene motifs; Holodeck uses language-proposed relations and constraints; I-Design uses specialized role prompts; LayoutVLM combines visual prompting and differentiable optimization; SceneWeaver iterates reasoning and visual reflection. LayoutVLM receives an added open-vocabulary asset extension with either its curated library or Objathor/Objaverse. Released models and tuning differ across systems.
M.1.2 Ablations
Ablations remove critique iterations, generated assets, VLM asset validation, six specialized placement tools, visual observations, or previous-turn designer/critic memory. Routing remains when validation is removed. The retrieved-only condition uses HSSD in the otherwise SceneSmith pipeline, partially isolating asset generation from scene composition.
M.1.3 Prompt Corpus
All 210 prompts are retained: 94 SceneEval room prompts, 50 new room types, 25 object-density prompts, 10 themes, 6 SceneEval house prompts and 25 additional houses. Thus room evaluation uses 179 scenes and house evaluation 31. The heterogeneous categories test different strengths rather than a naturally sampled deployment distribution.
M.2 SceneEval Metrics
SceneEval measures object count (CNT), attributes (ATR), object relations (OOR), object–architecture relations (OAR), functional accessibility (ACC), navigability (NAV) and out-of-bounds objects (OOB). Several rely on VLM interpretation. NAV is largest connected free-floor area divided by total free area, not robot task success; OOB uses fewer than 99% downward floor-ray hits.
M.2.1 Physics Metrics for Simulation Readiness
To simulate baseline outputs, add 128-hull V-HACD collision geometry and uniform 1,000 kg/m³ density for mass/inertia; exclude carpets. This differs from SceneSmith’s VLM physical estimates and repair pipeline. COL counts objects with penetration over 1 mm. STB requires under 1 cm translation and 0.1 rad rotation after five seconds of gravity at 1 ms steps. Fixtures are welded, with a VLM assigning support class for every method; classification errors can affect comparisons.
Appendix N Human Evaluation
Human pairwise judgments use two-sided binomial tests against 50%, Wilson intervals and Benjamini–Hochberg correction. The source states 28 tests for 14 comparisons, while Table 1 displays 13 comparison rows; this count is not reconciled. Reported power targets wins of at least 63%; effects around 53–55% would need many more comparisons. Participant-level dependence is not explicitly modeled.
N.1 Study Setup
Recruit 210 U.S. Prolific participants; 205 supply complete data, yielding 3,051 valid scene comparisons. Each participant sees 15 pairs and receives $12.50, with about 32 minutes mean completion time. Faithfulness ties are excluded from its win-rate denominator, so realism and faithfulness can have different sample counts.
N.2 Interface Design
Twelve independent testers informed the interface. Participants can orbit/pan/zoom a Babylon.js view, hide/select objects and switch to nine-angle carousel or grid views. Method names are hidden, scene positions randomized, and compressed models/lazy loading improve access on varied devices. Figures 20–23 preserve the interface rather than reconstructing it.
N.3 Questions
Two questions separate visual realism from explicit prompt requirements. The exact question, hint, tooltip and welcome-page wording is retained below because it defines the judgment being measured.
Q1: Realism
Realism is forced choice between scenes A and B, without a tie option. Guidance asks about placement, size and coherent, plausible room layout; it is not a physical-simulation test.
(forced choice: Scene A / Scene B)
“Which scene looks more realistic?”
Hint: “Consider: realism, object placement, layout coherence”
Tooltip: “Consider the realism of each scene: Does it look like a realistic room? Are the objects and furniture well-placed and appropriately sized? Does the overall layout feel coherent and plausible?”
Welcome page: “Which scene looks more realistic? Consider whether the room looks realistic, objects are well-placed and appropriately sized, and the layout feels coherent and plausible.”
Q2: Prompt Faithfulness
Faithfulness offers A, B or Equal, focuses on requested objects, room type and specified details, and permits additional objects. Selecting Equal triggers a confirmation encouraging reconsideration. That asymmetric tie treatment and exclusion of ties should accompany interpretation of its win rates.
(three options: Scene A / Scene B / Equal)
“Which scene better follows the requirements in the text description?”
Hint: “Consider: requested objects, room type, described details”
Tooltip: “Consider how well each scene matches the specific requirements mentioned in the prompt: Does the scene include the requested objects? Is the room type correct? Are details like colors, quantities, and arrangements as described? Additional objects that fit the room are fine.”
Welcome page: “Which scene better follows the requirements in the text description? Consider whether the requested objects are present, the room type is correct, and details like colors, quantities, and arrangements match what was described. Additional objects that fit the room are fine.”
Welcome page example: “If the prompt says ‘A living room with a sofa and a coffee table with a book on it’, the scene should include a sofa and a coffee table with a book on top. Additional items typical of a living room (like a TV, rug, or lamp) are fine. We don’t expect an empty room with just the mentioned objects.”
Equal confirmation: When selecting “Equal,” participants saw: “Are you sure both scenes follow the description equally well? Select ‘Equal’ only if you genuinely cannot distinguish which scene better matches the prompt.”
N.4 Randomization and Counterbalancing
Randomize scene side, comparison order and question order; balance baseline assignments; avoid showing the same prompt twice to one participant. These measures reduce positional, sequence and familiarity biases, while repeated judgments by a participant still share that person’s preferences.
N.5 Results by Prompt Category
Against external baselines, room-category realism wins range from 89.8% for themes to 94.7% for density. Faithfulness ranges from 89.7% for density to 94.9% for themes. Category totals combine baselines and vary in sample count; house results remain separate.
| Category | Realism Win% | Faith. Win% | (Real.) | (Faith.) |
|---|---|---|---|---|
| SceneEval-100 | 92.2% | 92.0% | 831 | 808 |
| Type Diversity | 90.3% | 89.8% | 309 | 304 |
| Object Density | 94.7% | 89.7% | 151 | 146 |
| Themed Scenes | 89.8% | 94.9% | 59 | 59 |
Appendix O Automated Evaluation
Automated evaluation reports semantic metrics, short simulation checks and generation resource usage. These measurements complement human preferences but sometimes miss the ablation differences visible to participants.
O.1 SceneEval Benchmark Results
The full method’s room scores are CNT 82.9, ATR 74.4, OOR 67.6, OAR 80.6, ACC 83.4, NAV 97.6, COL 1.2, STB 95.6 and OOB 0.2. Intervals are scene-level t intervals. Retrieved-only assets worsen CNT, OOR and stability, but many other ablation intervals overlap.
NoSpecializedTools and NoCritic even have somewhat better COL/STB point estimates than full, without a demonstrated significant advantage. Automated metrics do not reproduce every user-study preference. House SceneSmith/Holodeck object counts are 214.1/81.3, STB 79.8/17.9 and NAV 96.5/99.6.
Original selectable source
| Method | #Obj | CNT | ATR | OOR | OAR | ACC | NAV | COL | STB | OOB |
|---|---|---|---|---|---|---|---|---|---|---|
| External Baselines | ||||||||||
| HSM | 22.72.6 | 60.64.2 | 61.57.3 | 30.65.7 | 66.26.6 | 88.91.7 | 99.40.4 | 20.63.0 | 45.23.9 | 5.51.1 |
| Holodeck | 23.01.4 | 44.24.3 | 44.47.2 | 16.84.6 | 38.07.1 | 84.91.4 | 99.60.2 | 12.32.3 | 31.92.7 | 0.80.4 |
| IDesign | 13.01.0 | 69.34.0 | 50.37.2 | 28.65.5 | 66.66.7 | 70.12.8 | 95.91.4 | 3.01.5 | 60.84.6 | 4.32.0 |
| LayoutVLM (Curated) | 11.21.6 | 41.04.0 | 25.66.6 | 14.14.0 | 22.46.1 | 93.81.9 | 99.70.2 | 25.94.4 | 19.43.6 | 6.22.1 |
| LayoutVLM (Objaverse) | 14.21.7 | 55.54.7 | 34.37.0 | 20.75.0 | 17.45.6 | 91.51.6 | 98.70.8 | 28.93.5 | 8.11.7 | 5.11.4 |
| SceneWeaver | 13.51.0 | 41.83.9 | 29.16.7 | 15.34.2 | 31.66.5 | 77.52.7 | 98.11.2 | 12.52.5 | 37.34.4 | 0.00.0 |
| Ablations | ||||||||||
| NoCritic | 54.08.3 | 83.53.0 | 72.46.3 | 64.55.8 | 73.56.2 | 85.81.6 | 99.00.8 | 0.80.4 | 96.11.5 | 0.10.1 |
| NoObserveScene | 69.715.4 | 81.13.0 | 75.96.0 | 59.65.8 | 71.76.7 | 84.71.7 | 98.80.8 | 0.90.4 | 95.31.5 | 0.00.0 |
| NoAgentMemory | 78.919.7 | 82.53.1 | 72.66.5 | 66.05.8 | 81.55.3 | 84.51.6 | 97.61.2 | 1.70.7 | 94.11.9 | 0.10.1 |
| NoSpecializedTools | 61.514.1 | 83.12.9 | 75.26.1 | 66.05.5 | 81.25.4 | 83.81.5 | 97.71.1 | 0.50.3 | 97.61.2 | 0.00.0 |
| NoAssetValidation | 72.716.3 | 81.63.0 | 72.45.9 | 65.55.5 | 79.55.4 | 83.61.4 | 97.51.2 | 0.80.4 | 95.61.5 | 0.10.1 |
| NotGenerated | 57.79.9 | 75.73.1 | 64.26.9 | 53.76.2 | 72.46.3 | 83.91.6 | 97.31.3 | 7.41.3 | 88.42.4 | 0.10.1 |
| SceneSmith (Ours) | 71.113.0 | 82.93.4 | 74.46.2 | 67.65.8 | 80.65.5 | 83.41.5 | 97.61.1 | 1.20.6 | 95.61.7 | 0.20.4 |
O.2 Base Model Swap Control
Replacing HSM/Holodeck’s model with GPT-5.2 high reasoning while retaining prompts/tools gives mixed changes: some semantic scores improve while accessibility or physical scores can degrade. SceneSmith retains large advantages on several metrics. The control argues against base model alone explaining all gains, but it is not a fully retuned equal-compute architecture comparison.
Original selectable source
| Method | #Obj | CNT | ATR | OOR | OAR | ACC | NAV | COL | STB | OOB |
|---|---|---|---|---|---|---|---|---|---|---|
| HSM | 22.72.6 | 60.64.2 | 61.57.3 | 30.65.7 | 66.26.6 | 88.91.7 | 99.40.4 | 20.63.0 | 45.23.9 | 5.51.1 |
| HSM-GPT-5.2 | 31.22.9 | 63.84.1 | 57.87.3 | 37.16.5 | 66.36.7 | 83.32.1 | 99.10.6 | 31.83.5 | 36.74.0 | 7.21.1 |
| Holodeck | 23.01.4 | 44.24.3 | 44.47.2 | 16.84.6 | 38.07.1 | 84.91.4 | 99.60.2 | 12.32.3 | 31.92.7 | 0.80.4 |
| Holodeck-GPT-5.2 | 27.61.5 | 45.24.1 | 44.97.3 | 15.64.3 | 33.66.5 | 80.41.4 | 99.70.1 | 10.81.9 | 27.42.5 | 0.40.2 |
| SceneSmith (Ours) | 71.113.0 | 82.93.4 | 74.46.2 | 67.65.8 | 80.65.5 | 83.41.5 | 97.61.1 | 1.20.6 | 95.61.7 | 0.20.4 |
O.3 Equilibrium Statistics
Mean per-scene average displacement is 12.3±4.5 mm; mean per-scene maximum is 0.35±0.13 m. Thus a high stable fraction can coexist with large movement of a few objects. The prose’s “within about 1 cm” is looser than the 12.3 mm point estimate. NoAgentMemory has 795.1±1078.8 mm mean displacement, indicating severe outliers and an unbounded symmetric interval rather than possible negative physical displacement.
Original selectable source
| Method | MD (mm) | XD (m) | MR (rad) |
|---|---|---|---|
| SceneSmith + Ablations | |||
| SceneSmith (Ours) | 12.34.5 | 0.350.13 | 0.0140.004 |
| NoCritic | 28.534.0 | 1.311.40 | 0.0110.005 |
| NoObserveScene | 61.983.7 | 1.141.31 | 0.0200.011 |
| NoAgentMemory | 795.11078.8 | 1.921.94 | 0.0200.006 |
| NoSpecializedTools | 22.624.4 | 0.670.83 | 0.0070.003 |
| NotGenerated | 45.319.4 | 0.940.59 | 0.0490.015 |
| NoAssetValidation | 33.331.5 | 1.191.31 | 0.0110.003 |
| Baselines | |||
| HSM | 654.8164.1 | 5.021.88 | 0.1930.037 |
| Holodeck | 932.2233.6 | 7.952.69 | 0.2170.032 |
| IDesign | 382.3106.1 | 2.060.50 | 0.1140.027 |
| LayoutVLM (Curated) | 684.5117.1 | 2.490.46 | 0.2080.040 |
| LayoutVLM (Objaverse) | 906.3114.7 | 3.510.44 | 0.2990.049 |
| SceneWeaver | 575.8103.2 | 3.300.55 | 0.1930.033 |
O.4 Collision Depth Statistics
Mean penetration depth is conditional on colliding objects within each scene: 3.77±1.28 mm for full SceneSmith, not an average over all objects. Table 17 repeats COL values inconsistently for several baselines: LayoutVLM curated/Objaverse are 26.1/31.4 here versus 25.9/28.9 in Tables 2/14; SceneWeaver is 12.6 versus 12.5. Original values are preserved rather than silently reconciled.
Original selectable source
| Method | COL (%) | MPD (mm) |
|---|---|---|
| SceneSmith + Ablations | ||
| SceneSmith (Ours) | 1.20.6 | 3.771.28 |
| NoCritic | 0.80.4 | 3.031.44 |
| NoObserveScene | 0.90.4 | 4.443.67 |
| NoAgentMemory | 1.70.7 | 3.991.27 |
| NoSpecializedTools | 0.50.3 | 4.483.13 |
| NotGenerated | 7.41.3 | 6.141.91 |
| NoAssetValidation | 0.80.4 | 2.320.70 |
| Baselines | ||
| HSM | 20.63.0 | 14.543.46 |
| Holodeck | 12.32.3 | 12.943.88 |
| IDesign | 3.01.5 | 47.0525.31 |
| LayoutVLM (Curated) | 26.14.4 | 15.004.01 |
| LayoutVLM (Objaverse) | 31.43.5 | 12.781.78 |
| SceneWeaver | 12.62.6 | 10.873.21 |
O.5 Generation Cost and Time
Reported historical mean cost is $13.98 per room (SD $9.98, range $4.59–101.93); mean generation time is 3 h 26 min (SD 1 h 57 min, range 1 h 17 min–15 h 45 min) under 25 concurrent scenes on eight L40S GPUs. Designer calls account for 75.9% of cost and critics 20.0%.
NoCritic costs $4.24 and takes 1 h 40 min, with about 24% fewer objects. NoObserveScene costs $9.62 and takes 2 h 10 min; removing memory instead increases cost to $16.41 and time to 4 h 14 min. These are pipeline-specific cost/density tradeoffs, not current price quotations or an exhaustive deployment-cost analysis.
| Method | Mean | Std | Min | Max |
|---|---|---|---|---|
| Cost per scene ($) | ||||
| SceneSmith (Ours) | 13.98 | 9.98 | 4.59 | 101.93 |
| NoCritic | 4.24 | 2.15 | 1.73 | 22.70 |
| NoObserveScene | 9.62 | 7.46 | 3.94 | 75.56 |
| NoSpecializedTools | 14.31 | 9.48 | 5.07 | 72.54 |
| NoAssetValidation | 14.85 | 9.24 | 5.52 | 81.99 |
| NoAgentMemory | 16.41 | 10.76 | 5.56 | 96.62 |
| NotGenerated | 15.70 | 9.99 | 4.42 | 104.79 |
| Time per scene (hours:minutes) | ||||
| SceneSmith (Ours) | 3:26 | 1:57 | 1:17 | 15:45 |
| NoCritic | 1:40 | 1:04 | 0:42 | 12:17 |
| NoObserveScene | 2:10 | 0:49 | 0:48 | 5:24 |
| NoSpecializedTools | 3:14 | 1:41 | 1:20 | 17:43 |
| NoAssetValidation | 3:11 | 1:52 | 1:19 | 20:22 |
| NoAgentMemory | 4:14 | 2:01 | 1:48 | 15:35 |
| NotGenerated | 3:33 | 1:49 | 1:11 | 13:55 |
| Component | Cost ($) | % of Total |
|---|---|---|
| Designer | 10.61 | 75.9% |
| Critic | 2.79 | 20.0% |
| Orchestrator | 0.24 | 1.7% |
| Other | 0.34 | 2.4% |
| Total | 13.98 | 100% |
Appendix P Evaluation Prompts
The exact 210 source prompts are retained in six groups. They are evaluation inputs, not shortened examples or replacement summaries.
P.1 SceneEval-100 (Room) ( Tam et al., 2025 ) (94 prompts)
The 94 SceneEval room prompts span bedrooms, living/dining rooms and additional settings with positive, negative, quantity and spatial constraints. Their exact wording follows.
-
A bedroom with a bed, two nightstands, and a wardrobe in the corner of the room.
-
A bedroom with a bed and a wardrobe.
-
A bedroom with a king-size bed in the corner of the room, two large blue armchairs, and a floor lamp near a armchair.
-
A bedroom with a double bed and a mini fridge near the bed, a table across from the door, and a painting on the wall above the bed.
-
A bedroom with two bunk beds in the corners of the room, a desk placed against the wall far from the door, and a light hanging from the ceiling in the center of the room.
-
A simple bedroom featuring a twin bed against the wall, with a wardrobe positioned in the corner of the room, and a desk next to the window.
-
A minimal bedroom with a queen bed against the wall, flanked by two nightstands on each side.
-
A living room with a TV, sofa, and bookshelf. There is no coffee table in the room.
-
A living room with a TV, sofa, bookshelf, and coffee table.
-
A living room with two bookcases along the wall far from the door, a wooden chair near the bookcases, and a painting on the wall opposite the bookcases.
-
A living room with a two-seater sofa against the wall, a square rug in the middle in front of the sofa, and two large plants on the floor near the sofa.
-
A living room with a bean bag near a wall cabinet, which is across from the window. There is a stool next to the bean bag.
-
A living room with a sofa against the wall across from the door, a painting on the wall above the sofa, and a pendant light hanging in the middle of the room.
-
A dining room with two bar stools at the short sides of a bar table.
-
A dining room with three dining chairs surrounding a dining table.
-
A small dining room with a dining table next to a window, a chair on the long side of the table, and a rug in the middle of the room.
-
A dining room with a cabinet next to the door against the wall, and two wine cabinets against the wall near a bar table.
-
A dining room with a table in the corner of the room and a chair on the long side.
-
A dining room with a ceiling light hanging above the table and two chairs on the long side of the table.
-
A bedroom with a bed with two nightstands on each side, a wardrobe, and a TV stand positioned in front of the bed. There is a toy box with two dolls outside the box.
-
A bedroom with a bed flanked by two nightstands, and an office chair in front of an office desk located in the corner of the room. There is no wardrobe in the room.
-
A bedroom with a bed against the wall and at least one nightstand next to it, featuring a lamp on the nightstand. There is an office desk with a monitor on top and a sofa chair in front of it.
-
A teenager’s bedroom featuring a twin bed with an adjacent nightstand, a wardrobe for clothes, a small desk with a chair, and a bookshelf positioned next to the wardrobe. There are two books outside the bookshelf.
-
A teenager’s bedroom designed for both comfort and functionality includes a twin bed with an adjacent wooden nightstand, a large wardrobe with multiple drawers for clothes, and a small desk and chair equipped with an electronic setup featuring a monitor and peripherals. A large oak bookshelf is positioned next to the wardrobe.
-
A bedroom with a king-size bed positioned against the wall across from the window. Two stools are placed in front of the bed. A painting hangs above the bed, flanked by a wall light on each side. There is a bookcase with two books on the left side.
-
A bedroom with three extendable beds, one of which has a doll placed on top. Two of the beds are positioned in the corners of the room, and a dresser is located next to the door. There are two dolls outside a toy box.
-
A living room featuring two bookshelves along the wall, with only one book on the bookshelf nearest the door. Across from the bookshelves, a painting is mounted on the wall. In the center of the room, a table with a tray holding a potted plant sits near an L-shaped sofa.
-
A living room with a marble table in front of a four-seater sofa. The sofa is positioned in the corner of the room, with two bean bags placed next to the table. A lamp hangs from the ceiling.
-
A large living room with three display shelves against the wall, with a long sofa in front. There are two tables in front of the sofa and another table right of the sofa with a lamp on top. There is no TV in the room.
-
A Japanese-style living room featuring a coffee table next to the window with two floor cushions placed beside it. A sofa is positioned across from the window, and in front of the sofa is a table with a teapot on top, completing the serene and minimalist setup.
-
A living room featuring an irregular-shaped table in the middle of the room with a sofa positioned in front of it. Across the table are two sofa chairs with a small wooden coffee table placed between them. A clock is mounted on the wall far from the door.
-
A dining room with a bar table positioned in the middle of the room. A wooden shelf is mounted on the wall, holding a potted plant and a jug, adding a touch of decor and functionality. Along the wall, there is a fridge and a wine cabinet, providing ample storage and a cohesive design.
-
A dining room with a fridge positioned near the door against the wall. A table is placed in the middle of the room with two chairs on its long sides. In the corner of the room across from the door, there is a cabinet.
-
A dining room with six wooden dining chairs surrounding a round wooden table in the middle of the room. There is no coffee table in the room.
-
A dining room with a square dining table, featuring two chairs on each long side and one chair on each short side of the table.
-
A dining room with a dining table featuring four chairs on the long side and one on a short side. A vase is placed in the middle of the table. A display cabinet is positioned next to a window against the wall.
-
Generate a dining room. The room should have a rectangular, wooden dining table with three chairs surrounding it. There is a utility cart near the table with three plates inside.
-
A spacious master bedroom featuring a king-size bed against the wall, flanked by two nightstands, one holding a modern table lamp and a small book. A large wardrobe is positioned in the corner of the room, and a wooden dresser with three picture frames on it. Near the window, a plush armchair with a throw blanket provides a cozy seating area. A round area rug is placed at the foot of the bed, adding warmth and style to the space.
-
A teenager’s bedroom features a comfortable twin bed with a backboard in the far corner, with boxes underneath it. At the foot of the bed is a small desk equipped with a monitor, an external keyboard and mouse, and a desk lamp on the right for visibility, accompanied by a rolling chair. Next to the bed, a nightstand with an additional floor lamp nearby provides space for a phone and other valuables. A sizable wooden wardrobe with multiple drawers offers ample storage for clothes, while a coffee table beside it holds books and board games. In the center of the room, a tan-colored rug creates a cozy spot to sit, and the walls are adorned with various posters and pictures.
-
A bedroom featuring a bunk bed positioned next to the window, adorned with plush toys on the bed. The room includes two desks placed against the wall, each accompanied by a chair. Over ten plush toys are scattered across the desks, adding a playful and cozy touch to the space.
-
A modern living room featuring an L-shaped sofa facing a wall-mounted TV above a floating media cabinet. Two leather accent chairs face a glass coffee table in the center, which displays two art books. A bar cart stocked with glasses and bottles stands in the corner, while three pendant lights hang gracefully from the ceiling. The sofa is flanked by two side tables, one of which holds a table lamp, and a large abstract painting decorates the wall.
-
The room is dimly lit, creating a somber atmosphere in a cozy and well-furnished living room. In the center of the room, there is a dining table with four wooden dining chairs arranged neatly around it. The table appears to be set, indicating a recent meal or gathering. Adjacent to the dining area, There is a glass coffee table serving as a centerpiece for the seating arrangement. It is both stylish and practical, providing a surface for drinks or decor. A multi-seat sofa is in front of the coffee table, providing ample seating for guests. Completing the seating options, two comfortable armchairs also face the coffee table on opposite sides near the sofa. Next to the sofa and each armchair is a corner side table, adding a touch of functionality and convenience. Each table has a lamp on it for lighting. The sofa and seating arrangement should face an opposite wall, against which someone in the scene could view a large flatscreen TV. The TV is supported by a TV stand with storage underneath for various books. The room appears to reflect the inhabitants’ taste for a warm and inviting environment, despite the unsettling news program playing on the TV in the background. The juxtaposition of the serene living room with the chaos and screams on the TV screen creates a sense of tension and unease.
-
A modern living room featuring four sofa chairs surrounding a circular ceramic table, with two floor lamps positioned adjacent to the table. All the furniture is set on a large rectangular rug in the middle of the room. A large wooden tray with two glass bowls rests on top of the table. A mirror is mounted on the wall next to the window, flanked by a wall lamp on each side.
-
An open-plan dining area featuring a round table surrounded by six mid-century modern chairs. A grand chandelier illuminates the table, while a sideboard with four drawers rests against the wall, topped with three potted plants. Two wall sconces provide additional lighting, flanking a large mirror that hangs between them. A wine cabinet is positioned in the corner, with a serving cart placed nearby for added convenience.
-
A cozy bedroom featuring a queen-sized bed against a soft pastel wall, with a small desk and bookshelf tucked neatly into the corner.
-
A simple bedroom featuring a full bed flanked by two bedside lamps, with a large wardrobe positioned directly facing the bed.
-
A bright bedroom with a king-sized bed, a plush area rug under it, and a dresser along the wall.
-
A large bedroom with a California king bed, a reading table in the corner with a comfy chair, and a wall-mounted TV for entertainment.
-
A living room featuring an overstuffed sofa, a vintage wooden table, alongside a bicycle hung decoratively on the wall.
-
A chic living room with a bright rug, a gallery wall of family photos, and a comfortable hammock chair.
-
A contemporary living room with a leather sofa, a small coffee table, and a vintage typewriter.
-
A sunny dining room with a large picnic-style table, a colorful tablecloth, and a lovely herb garden next to the window.
-
A simple dining room featuring a farmhouse table, a rustic chandelier, and a small cart placed against the wall.
-
A minimalist dining room with a glass table and an industrial-style metal chair next to it, accented by a unique wall-mounted wine rack.
-
In a cheerful playroom, a spacious foam play mat lies in the middle, with a small climbing structure positioned nearby for active play, and a cozy bench situated against the wall.
-
This playroom features a large colorful rug at its center, with a play kitchen set against one wall and a rocking horse placed on the rug.
-
A minimalist gaming room features a wall-mounted screen paired with a low-profile media console below, while a compact gaming desk with an ergonomic chair is positioned nearby.
-
A kitchen featuring a bistro table with two chairs in a corner and a refrigerator positioned nearby.
-
A stylish kitchen with an elegant pendant light hanging over a marble-topped kitchen counter. A single cabinet is positioned against the wall, providing ample storage space.
-
A bathroom with a freestanding bathtub positioned beside a frosted window and a double vanity located across from it.
-
In a classic bathroom, a double-sink vanity is set against one wall with two round mirrors hanging above, while a toilet is positioned in the corner.
-
In this basement, a pool table occupies the center of the room, with two bar stools lined up at a nearby bar table, allowing for social gatherings and entertainment.
-
A gym-purpose basement featuring a treadmill and an exercise bike positioned against one wall, with a tall mirror mounted on a wall directly facing the exercise bike.
-
This is a small, plain bedroom. Upon entering through the door, a white desk with a black rolling chair in front of it is positioned against the wall to the right. A queen bed is centered against a wall, with two modern nightstands beside it, one of which holds a stylish lamp.
-
A large bedroom with a lofted twin bed in one corner, freeing up the floor space below for a comfy armchair and a small side table. A bookcase stands against the wall next to the armchair, while a small desk, equipped with a pen and some papers along with a lamp for studying, is positioned near the window.
-
A bedroom with a modern queen bed against the main wall. Two bedside tables stand on either side, while a small writing desk is positioned in the corner next to the window, equipped with a cute desk lamp. A wicker basket lies at the foot of the bed.
-
A bedroom featuring a queen-sized bed against one wall, with a whimsical painting hung above it. A small bedside table sits to the left of the bed, holding a cute alarm clock, while a dresser stands across from the door.
-
A living room with a sectional sofa, a wooden coffee table in the middle, and a retro record player in the corner with some records nearby.
-
A living room with a large wall-mounted screen, several chairs arranged in front of it, and a collection of board games on a shelf in the corner.
-
A cozy living room with an L-shaped oversized sofa against one wall, adorned with three colorful pillows. In front of the sofa, a wooden coffee table holds a small vase with fresh flowers.
-
A dining room featuring a simple rectangular table, a corner shelf filled with cookbooks, and a pendant light hanging from the ceiling.
-
A simple dining room with a picnic table, illuminated by string lights overhead, and a small cooler holding utensils and napkins.
-
A dining room with a round wooden table in the middle of the room, surrounded by four wooden chairs, and a large vintage map displayed on the wall.
-
A bright playroom featuring a low table with board games in the center, surrounded by three small chairs for group activities, while an adjacent puppet theater is positioned against the wall.
-
This engaging playroom features a small painting stand positioned next to the window, a bookshelf full of toys standing against the nearby wall, and a cozy reading chair set across from the door.
-
A tech-styled gaming room featuring a glass gaming desk with dual monitors and a powerful gaming PC, with an ergonomic chair positioned directly in front.
-
A welcoming gaming room featuring a U-shaped sofa with a coffee table in front, facing a large TV on the wall. In the corner, there is a mini arcade machine and a PlayStation.
-
In this modern kitchen, a large bar table with four high-backed stools stands in the center, facilitating social gatherings, while a stainless steel fridge is positioned against the far wall, next to a sleek pantry shelf. A small bowl of fruit sits on the table’s surface.
-
This cozy kitchen features a rustic wooden dining table positioned against the window, surrounded by four chairs. A sideboard against the wall is equipped with a coffee maker and a small herb planter.
-
In this bathroom, a wide vanity sits against the wall, paired with a tall mirror mounted above. Nearby, a stylish laundry hamper is positioned next to the door, while a small basket containing bath essentials rests beside the tub.
-
A bright bathroom features a spacious bathtub positioned beside a large window, while a stylish shelving unit stands against the wall, displaying towels and small decorative items. The toilet is placed across from the tub, with a small basket beside it.
-
This versatile basement features a large sofa and an area rug in the center, creating a cozy movie area, while a small treadmill is positioned in the corner for quick exercise sessions. A mini chalkboard hangs on the wall next to a wall clock.
-
This entertaining basement layout features a large gaming setup with two monitors on a desk against one wall, while a comfy bean bag chair is positioned nearby for casual seating. Across from the gaming area, a small cabinet with a mini fridge and a popcorn machine on top completes the setup.
-
A welcoming bedroom with a full bed placed against the wall, flanked by two nightstands topped with a bedside clock and a decorative candle. A reading chair sits in the corner next to a tall bookshelf filled with novels.
-
A bedroom featuring a vintage double bed dressed in soft linens, with a painting hanging on the wall above it. On either side of the bed are wooden nightstands, each with a small decorative vase. A tall chest of drawers stands against the opposite wall. A small round table in the corner holds a lamp, colorful cobblestones, and photo frames.
-
A living room featuring a comfortable recliner chair placed next to a tall bookshelf filled with books and small sculptures. Across from the chair, a table next to a large window holds a collection of candles. In the center of the room, a round coffee table is surrounded by a simple sofa and two cube ottomans.
-
A living room with a comfortable loveseat against the wall, adorned with plush cushions. Next to the loveseat, a small table holds a cup of tea and a remote control. A mid-century style coffee table sits in front of the loveseat with art magazines spread out. A large bookshelf containing books and newspapers lines the wall across from the door.
-
A dining room with a circular table surrounded by six vintage chairs, and an old wooden ladder against the wall displaying plants and decorative jars.
-
In this playroom, a large toy chest sits against the wall, overflowing with colorful building blocks, while a small craft table nearby is equipped with papers, crayons, and glue sticks for hands-on projects.
-
This gaming room features powerful speakers positioned in every corner to create an immersive audio experience, with a large screen mounted opposite a plush sofa. A small table beside the sofa holds various game controllers.
P.2 Type Diversity (50 prompts)
Fifty prompts extend room-type diversity to stores, offices, clinics, workshops, hospitality and other interiors. Each prompt is retained verbatim.
-
A pet store with aquariums along one wall, exactly two shelf units where one displays pet food bags and the other displays pet toys, pet beds on the floor, and a checkout counter near the entrance.
-
A bookstore with at least four tall bookshelves, each containing at least five books. A central display table showcases featured titles, and two comfortable reading chairs sit in one corner.
-
A small grocery store with two refrigerated display cases along one wall. Two tables near the entrance hold produce bins, each containing at least three fruits. At least four shopping baskets are stacked by the door.
-
A pharmacy with a pharmacy counter in the back of the room. Two shelves stand against the walls, each holding at least four medicine bottles. Three waiting chairs are arranged near the counter.
-
A toy store with two large toy bins in the middle of the room, each containing at least five stuffed animals. Three shelves along the walls display multiple board games each, and a rocking horse sits in one corner.
-
An electronics store with two display tables in the center, one showcasing three laptops and the other holding four tablets. Two large televisions are mounted on the wall, and a headphone display stand holds headphones.
-
A furniture showroom with a sofa and coffee table arrangement in one corner, a dining table surrounded by four chairs in the center, and two floor lamps positioned near the walls. A large rug lies under the dining set.
-
A flower shop with three flower buckets on the floor, each holding flowers. A wooden table displays potted plants. A potted bonsai tree sits in one corner, and two Christmas trees stand in another corner.
-
A hotel lobby with a reception desk in the back of the room. A seating area in the center features two sofas facing each other with a coffee table between them. A luggage cart stands near the entrance, and a large chandelier hangs from the ceiling.
-
A hotel room with a queen bed flanked by two nightstands, a desk with a chair, and a luggage rack in the corner.
-
A restaurant with four dining tables, each surrounded by four chairs. A bar counter runs along one wall with six bar stools in front. Two potted plants flank the entrance, and pendant lights hang above each table.
-
A cafe with a bar counter along one wall featuring a coffee machine, a filter coffee stand, and stacks of stone coffee cups. Three small round tables with two chairs each are arranged in the center. A chalkboard menu hangs on the wall behind the counter.
-
A bar with a long wooden bar counter and five bar stools. Shelves behind the counter display bottles. Two high tables with two stools each are near the window.
-
A diner with three booth tables against the wall, a jukebox in the corner, and a hostess stand near the entrance.
-
A private office with a desk and office chair, a shelf with books against the wall, and two guest chairs facing the desk.
-
An open office with four desks arranged in pairs facing each other. Each desk has a monitor and an office chair. A water cooler stands in the corner.
-
A conference room with a long table surrounded by eight chairs. A projector screen hangs on the wall, and a whiteboard is mounted on another wall.
-
A reception area with a reception desk, at least three waiting chairs, and a potted plant in the corner.
-
A classroom with six student desks, each with a chair. A teacher’s desk sits at the front near the chalkboard, which hangs on the wall.
-
A library with shelves containing books, reading tables with chairs, and a commercial printer.
-
A computer lab with six desks, each with a monitor, keyboard, and chair. A printer sits in the corner.
-
An art studio with easels, stools, and a supply cabinet in the corner.
-
A music room with a piano, a drum set, and music stands with chairs.
-
A medical exam room with an exam table, a doctor’s stool, and a cabinet with medical supplies and a blood pressure monitor.
-
A hospital room with a hospital bed, an IV stand, a bedside table, and a visitor chair.
-
A dental office with a dental chair, a dentist’s stool, a dental light overhead, and a cabinet with dental tools.
-
A waiting room with chairs along the walls, a coffee table with magazines, and a water dispenser in the corner.
-
A physical therapy room with a treatment table, exercise mats on the floor, dumbbells, and a full-length mirror on the wall.
-
A gym with two treadmills and two exercise bikes along the wall. A weight rack holds dumbbells. A squat rack has five 45-pound weight plates stacked next to it. Exercise balls are in the corner, and a large mirror covers one wall.
-
A yoga studio with yoga mats on the floor, a mirror on the wall, and meditation cushions in the corner.
-
A dance studio with a ballet barre along the wall, a large mirror, and a speaker in the corner.
-
A spa treatment room with a massage table, a cabinet with towels, and candles on a round side table.
-
A robotics lab with two workbenches with robotic arms. Tool shelves on the wall hold tools, and a 3D printer sits in the corner.
-
A workshop with a wooden workbench, a stool, and a model airplane on the workbench. A tool box contains tools including a drill, hammer, screwdriver, and tape.
-
A garage with a car, a workbench, tires stacked in the corner, and a ladder against the wall.
-
A server room with three server racks, a fire extinguisher in the corner, and a desk with two monitors.
-
An art gallery with at least five paintings on the walls, a sculpture on a pedestal in the center, and a bench for viewing.
-
A commercial kitchen with a large stainless steel prep table, an industrial stove, a commercial refrigerator, and shelves with pots and pans. A utensil crock holds utensils including a spatula, tongs, and whisk. A cutting board with a chef’s knife on top sits on the table.
-
A bakery with a display case showing pastries, a counter with a cash register, and a bread rack with loaves of bread.
-
A nail salon with two manicure tables, each with a chair and a stool, and a shelf with nail polish bottles.
-
A barber shop with two barber chairs facing mirrors on the wall, and a waiting bench near the entrance.
-
A tattoo parlor with a tattoo chair, a rolling cart with ink bottles, and framed artwork on the walls.
-
A pet grooming salon with a grooming table, a bathtub, and a cage for pets in the corner.
-
A laundromat with four washing machines along the wall, two dryers, a folding table, and a row of plastic chairs.
-
A nursery with a crib, a rocking chair, a changing table, a baby mobile hanging from the ceiling, and a pile of wooden play blocks on the floor.
-
A daycare room with small tables and chairs, a storage box with toys in the corner, and colorful bins on a shelf.
-
A wine cellar with wooden shelves along the walls holding at least ten wine bottles. A tasting table with four stools sits in the center, with four wine glasses and a plate with cheese on it. A barrel is in the corner, and an antique chandelier hangs from the ceiling.
-
A science lab with two lab benches, each with a microscope, a glass beaker, and a test tube stand. A fume hood hangs from the ceiling, and a safety shower is in the corner.
-
An Egyptian museum room with a sarcophagus in the center. One display table holds a golden scarab beetle and a canopic jar. Another display table has an ancient scroll, a clay tablet, and a bronze amulet. Hieroglyphic panels are mounted on the walls, and two stone statues flank the entrance.
-
An escape room with a locked chest, a bookshelf with books, a clock on the wall, and a mysterious painting.
P.3 Object Density (25 prompts)
Twenty-five prompts stress dense arrangements and exact or minimum item counts, including filled shelves, containers and table settings. All numerical requirements remain in the source list.
-
A pantry with a single three-level shelf. The first level holds at least 10 cans, the second level holds at least 10 pasta packs, and the third level holds at least 10 potatoes.
-
A kitchen with a table. There is a fruit bowl on the table with a mix of apples and oranges. The bowl contains at least 6 fruits.
-
A kitchen with a single shelf. The shelf holds at least 8 plates, 8 bowls, and 8 cups.
-
A kitchen with a spice rack on the wall. The rack holds at least 10 spice jars.
-
A kitchen with an open refrigerator. The refrigerator contains at least 6 bottles and 6 food containers.
-
A kitchen. There is a utensil holder on the counter. The holder contains at least 4 forks, 4 knives, and 4 spoons.
-
A dining room with a table set for 4 people. Each setting has a plate, a glass, a fork, and a knife.
-
A dining room with a table set for 8 people. Each setting has a plate, a glass, a fork, and a knife.
-
A dining room with a long table set for 12 people. Each setting has a small plate stacked on a large plate, a glass, a fork, and a knife.
-
A kitchen with a mug rack on the wall holding at least 8 mugs.
-
A bedroom with a breakfast tray on the bed. The tray has a plate, a cup, a bowl, and a glass.
-
A living room with a coffee table. On the table is a tea set with a teapot, 4 teacups on saucers.
-
A bar with a counter. On the counter are at least 8 glasses and 6 bottles.
-
A dining room with a wine shelf holding at least 12 wine bottles.
-
A pottery store with shelves along the walls. The shelves hold at least 30 cups and 30 bowls. There is a table with a large painted bowl.
-
A bathroom with a counter. On the counter are at least 4 bottles, 2 tubes, and a soap dispenser.
-
A bathroom with a medicine cabinet. The cabinet contains at least 6 medicine bottles, 4 pill boxes, 2 tubes, and a pair of scissors.
-
A kitchen with a three-level shelf. The top level holds at least 10 jars. The other two levels hold stacks of plates with at least 20 plates in total.
-
A closet with a shoe rack holding exactly 5 pairs of shoes.
-
A bedroom with a vanity table. On the table are at least 6 makeup bottles, at least 2 brushes, and a small mirror. There is also a nightstand with a stack of 8 books.
-
An office with a desk. On the desk is a pen holder with at least 5 pens and a laptop. There is a shelf with stacks of books, at least 15 books in total.
-
A kitchen with a sink. The sink contains a mess of dishes with at least 5 plates, 3 glasses, and 5 bowls.
-
A playroom with two bins. One bin contains 2 teddy bears. The other bin contains at least 5 wooden blocks. There is a carpet with a toy car and a toy train. There is a pile of lego blocks on the carpet containing at least 5 blocks.
-
A restaurant with exactly 4 tables and 8 chairs. Each table is set for 2 people with a plate, a glass, a fork, and a knife.
-
A bookstore with at least 50 books.
P.4 Themed Scenes (10 prompts)
Ten prompts specify recognizable visual themes and detailed styling requirements. These assess prompt-conditioned appearance rather than physical validity.
-
A Star Wars themed teen bedroom. The room features a bed with Star Wars bedding showing characters and starships, and a Star Wars themed desk lamp with Darth Vader or Death Star designs.
-
A playroom inspired by The Incredibles movie. The room features an Incredibles themed rug with the superhero family logo and two Incredibles themed bean bags for seating.
-
An Art Deco styled living room. The room features an Art Deco style sofa with geometric patterns, an Art Deco style coffee table positioned in front of the sofa, and an Art Deco style floor lamp with a geometric shade.
-
A Minecraft themed gaming room. The room features a Minecraft themed chair with Creeper designs and a Minecraft themed poster on the wall.
-
A Picasso-inspired dining room. The room features a dining table with four chairs surrounding it. A Picasso-style cubist painting hangs on the wall, and a tablecloth with abstract geometric patterns covers the table.
-
A steampunk styled home office. The room features a steampunk desk with brass fittings and exposed gears, a steampunk style desk lamp with copper pipes and vintage bulb, a steampunk themed wall clock with visible clockwork mechanisms, and a steampunk styled office chair with leather and metal accents.
-
A Harry Potter themed home library. The room features a bookshelf styled like a Hogwarts library, a Harry Potter themed reading chair with a Harry Potter themed floor lamp next to it, and a Hogwarts trunk for storage.
-
A Pop Art styled artist studio. The room features a Pop Art style desk with bold colors, a Pop Art style stool positioned in front of the desk, a large Pop Art style painting with comic-style imagery on the wall, and a Pop Art themed floor lamp.
-
A Jurassic Park themed kids bedroom. The room features a bed with Jurassic Park dinosaur bedding, a Jurassic Park themed rug on the floor, and a Jurassic Park poster on the wall. Two dinosaur stuffed animals sit on the bed, and a dinosaur themed desk lamp rests on the nightstand.
-
A Frozen themed nursery. The room features a Frozen themed crib with Elsa and Anna bedding, a Frozen themed rocking chair, a Frozen themed rug with snowflake patterns on the floor, a Frozen themed lamp on a side table, a Frozen poster on the wall, and a Frozen themed toy box for storage.
P.5 SceneEval-100 (House) ( Tam et al., 2025 ) (6 prompts)
Six SceneEval multi-room prompts include bedrooms with attached storage, bathrooms or living spaces. They are kept separate from the 94 room-only SceneEval prompts.
-
A medium-sized bedroom featuring a queen-size bed with a nightstand on each side. A TV is mounted on the wall directly across from the bed, providing convenient viewing. The room includes a walk-in closet equipped with three wardrobes for clothes and two large mirrors.
-
A master bedroom with a king-size bed, an attached walk-in closet, and a master bathroom. A desk is positioned in one corner of the room for working, there is a laptop on the right side of the desk and a wallet on the left side. There are also two sofa chairs next to the window. There is a sink mounted in the bathroom with a soap on the left side.
-
The space includes three rooms: a spacious living room with an L-shaped sectional sofa facing a wall-mounted TV and a rectangular coffee table in front, holding a small serving tray with napkins. The dining room features an extending dining table surrounded by six chairs, a sleek sideboard with cups and sodas on top, two wall sconces providing soft lighting, and a small bar cart loaded with bottles in the corner. The modern kitchen boasts a central bar table with four bar stools on the long side, a bar table holding a charcuterie board with plates beside it, and a wine cooler in the corner.
-
This is a modern bedroom with a spa-like bathroom. The bedroom has a queen-size bed with vibrant pillows. A sleek dresser sits against one wall, adorned with a small jewelry box and a decorative mirror. A reading corner includes an armchair and a small side table. Adjacent to the bedroom, the bathroom features a large bathtub next to a frosted window. A vanity opposite the tub holds two sinks with skincare products.
-
This cozy bedroom features a full-size bed against a wall with two nightstands on each side where the left one has a small clock. A small desk sits in the corner with a comfortable chair for studying or working. Opposite the bed, a spacious dresser provides additional storage. Adjacent to the bedroom, the living room has a recliner and an ottoman facing a low coffee table with a small vase and magazines. A large bookshelf in the corner holds books and board games, while a wide couch provides space for gatherings. Just off the living room, a small gaming room with a gaming console and two beanbag chairs offers a dedicated space for entertainment, complete with a small shelf for controllers and headsets.
-
This bedroom has a full-size bed against a wall with a cartoon painting above. Two nightstands on each side of the bed provide storage. A writing desk in the corner holds a modern lamp for task lighting. A large wardrobe stands next to the door for additional storage. Beside the bedroom is a cozy living room with a large sofa facing a sleek coffee table where puzzles and decorative coasters reside. A bookshelf in the corner displays books and family pictures. The stylish bathroom includes a bathtub, with a toilet in the corner. The vanity features a double sink, with toiletries and white candles on top. A wicker basket at the foot of the bathtub holds towels.
P.6 House-Level (25 prompts)
Twenty-five additional multi-room prompts cover apartments, offices, commercial and medical spaces, and a detailed boutique hotel. All required rooms, contents and connectivity descriptions remain intact.
-
A compact studio apartment with a single open-plan room and a bathroom. The studio contains a bed against the wall, a sofa, and a desk with a chair. The bathroom has a toilet, a sink, and a blue bathtub.
-
A hotel room with a bedroom and an en-suite bathroom. The bedroom has a bed with two nightstands on each side, a desk with a chair, and a TV mounted on the wall. The bathroom has a toilet, a sink, and a shower.
-
A cozy guest cottage with a bedroom, a sitting area, and a bathroom. The bedroom has a bed with a nightstand and a lamp on top. The sitting area has two armchairs and a wooden bookshelf full of books. A skull sits on the bookshelf. The bathroom has a toilet and a sink.
-
A small office suite with a reception, a private office, and a bathroom. The reception has a reception desk with a computer on it and two chairs. The private office has an office desk with a chair and a filing cabinet. The bathroom has a toilet and a sink.
-
A massage parlor with a treatment room, a waiting area, and a bathroom. The treatment room has a massage table and a shelf with bottles. The waiting area has a sofa and a coffee table with books on it. The bathroom has a toilet and a sink.
-
A tiny house with a bedroom, a living-kitchen, and a bathroom. The bedroom has a bed and a wooden dresser. The living-kitchen has a sofa, a dining table with two chairs, and a refrigerator. The bathroom has a toilet and a sink.
-
A hair salon with a salon floor, a waiting area, a storage room, and a bathroom. The salon floor has two salon chairs positioned in front of two mirrors on the wall, facing them. The waiting area has a sofa and a coffee table. The storage room has shelves with bottles. The bathroom has a toilet and a sink.
-
A dorm suite with a bedroom, a study room, a common_room, and a bathroom. The bedroom has a bed and a wardrobe. The study room has a desk with a chair and a bookshelf with books. The common_room has a refrigerator and a microwave on a counter. The bathroom has a toilet and a sink.
-
A small retail shop with a showroom, a fitting room, a storage room, and a bathroom. The showroom has a display table with clothes on it and a cash register on a counter. The fitting room has a mirror and a bench. The storage room has shelves with boxes. The bathroom has a toilet and a sink.
-
A two-bedroom apartment with a master bedroom, a second bedroom, a living room, a hallway, and a bathroom. The master bedroom has a bed and a nightstand. The second bedroom has a bed and a desk with a chair. The living room has a sofa, a coffee table, and a TV. The hallway has a coat rack. The bathroom has a toilet, a sink, and a bathtub.
-
A boutique hotel suite with a bedroom, a living area, a walk-in closet, a dressing room, and a bathroom. The bedroom has a king-size bed with two nightstands. The living area has a sofa and a coffee table. The walk-in closet has a wardrobe and a luggage rack. The dressing room has a vanity with a mirror. The bathroom has a toilet, a sink, and a bathtub.
-
A small restaurant with a dining area, a kitchen, a bar, a storage room, and a bathroom. The dining area has four dining tables, each with two chairs. The kitchen has a stove and a refrigerator. The bar has a bar counter with three bar stools. The storage room has shelves with supplies including plates and cups. The bathroom has a toilet and a sink.
-
A small family home with a master bedroom, a kids room, a living-dining room, a kitchen, a hallway, and a bathroom. The master bedroom has a bed and a dresser. The kids room has a bunk bed and a toy box. The living-dining room has a sofa, a dining table with four chairs, and a TV. The kitchen has a refrigerator and a stove. The hallway has a coat rack and a shoe cabinet. The bathroom has a toilet, a sink, and a bathtub.
-
A dental office with a reception-waiting area, two exam rooms, an X-ray room, a corridor, and a bathroom. The reception-waiting area has a reception desk with a computer and three chairs. The first exam room has a dental chair and a cabinet. The second exam room has a dental chair and a sink. The X-ray room has an X-ray machine. The corridor has a bench. The bathroom has a toilet and a sink.
-
An art gallery with three exhibition rooms, an office, a storage room, and a bathroom. The first exhibition room has two paintings on the wall and a bench. The second exhibition room has three sculptures on pedestals. The third exhibition room has a painting on the wall with a bench in front for viewing. The office has a desk with a chair and a computer. The storage room has shelves with crates. The bathroom has a toilet and a sink.
-
A co-working space with an open office, two meeting rooms, a lounge, a kitchen, a reception, and a bathroom. The open office has four desks with chairs and computers. The first meeting room has a conference table with six chairs. The second meeting room has a whiteboard and four chairs. The lounge has two sofas and a coffee table. The kitchen has a refrigerator and a microwave on a counter. The reception has a reception desk with a computer. The bathroom has a toilet and a sink.
-
A photography studio with a shooting area, an editing room, a client lounge, a dressing room, a props storage, an office, and a bathroom. The shooting area has two studio lights and a backdrop. The editing room has a desk with two monitors and a chair. The client lounge has a sofa and a coffee table. The dressing room has a vanity with a mirror on top and a clothing rack. The props storage has shelves with props including hats and bags. The office has a desk with a chair and a filing cabinet. The bathroom has a toilet and a sink.
-
A small fitness studio with a workout area, a yoga room, a locker room, a reception, an office, a storage room, and a bathroom. The workout area has a treadmill, a weight rack with dumbbells, and a squat rack with a stack of weight plates next to it. The yoga room has yoga mats and a mirror on the wall. The locker room has lockers and a bench. The reception has a reception desk with a computer. The office has a desk with a chair. The storage room has shelves with towels. The bathroom has a toilet and a sink.
-
A pet clinic with a reception, a waiting room, two exam rooms, a surgery room, a corridor, and a bathroom. The reception has a reception desk with a computer. The waiting room has four chairs and a water dispenser. The first exam room has an exam table and a cabinet. The second exam room has an exam table and a scale. The surgery room has a surgery table and a surgical light. The corridor has a bench. The bathroom has a toilet and a sink.
-
A large family home with a master bedroom, two kids bedrooms, a living room, a kitchen, a dining room, a hallway, and two bathrooms. The master bedroom has a bed with two nightstands and a dresser. The first kids bedroom has a bed and a desk with a chair. The second kids bedroom has a bunk bed and a bookshelf with books. The living room has a sofa, a coffee table, and a TV. The kitchen has a refrigerator, a stove, and a dining table with four chairs. The dining room has a dining table with six chairs and a sideboard. The hallway has a coat rack and a mirror on the wall. The first bathroom has a toilet, a sink, and a bathtub. The second bathroom has a toilet and a sink.
-
A medical clinic with a reception, a waiting room, four exam rooms, a lab, a pharmacy, a corridor, and two bathrooms. The reception has a reception desk with a computer. The waiting room has six chairs and a water dispenser. The first exam room has an exam table and a cabinet. The second exam room has an exam table and a scale. The third exam room has an exam table and a desk with a chair. The fourth exam room has an exam table and a stool. The lab has a lab bench and a microscope. The pharmacy has shelves with bottles. The corridor has a bench. The first bathroom has a toilet and a sink. The second bathroom has a toilet and a sink.
-
A small office building with a reception, five offices, two conference rooms, a break room, a server room, a corridor, and a bathroom. The reception has a reception desk with a computer and two chairs. The first office has a desk with a chair and a computer. The second office has a desk with a chair and a filing cabinet. The third office has a desk with a chair and a bookshelf. The fourth office has a desk with a chair and a printer. The fifth office has a desk with a chair and a whiteboard. The first conference room has a conference table with eight chairs. The second conference room has a conference table with six chairs, a projector, and a whiteboard on the wall. The break room has a refrigerator, a microwave on a counter, and a table with four chairs. The server room has two server racks. The corridor has a water dispenser. The bathroom has a toilet and a sink.
-
A community center with a main hall, four activity rooms, a kitchen, an office, a reception, a locker room, a hallway, two bathrooms, and a storage room. The main hall has a stage and rows of chairs. The first activity room has tables and chairs for crafts. The second activity room has yoga mats and a mirror on the wall. The third activity room has a ping pong table. The fourth activity room has easels and stools for art class. The kitchen has a refrigerator, a stove, and a counter. The office has a desk with a chair and a computer. The reception has a desk with a computer. The locker room has lockers and a bench. The hallway has a bulletin board on the wall. The first bathroom has a toilet and a sink. The second bathroom has a toilet and a sink. The storage room has shelves with boxes.
-
A boutique hotel with a reception, four guest bedrooms with en-suite bathrooms, a restaurant, a kitchen, a gym, a lounge, and a corridor. The reception has a desk with a computer and two armchairs. The first guest bedroom has a bed with two nightstands and its bathroom has a toilet, a sink, and a shower. The second guest bedroom has a bed and a desk and its bathroom has a toilet, a sink, and a bathtub. The third guest bedroom has a bed and an armchair and its bathroom has a toilet, a sink, and a shower. The fourth guest bedroom has a bed and a wardrobe and its bathroom has a toilet, a sink, and a bathtub. The restaurant has four dining tables with chairs. The kitchen has a refrigerator, a stove, and a counter. The gym has a treadmill and a weight rack with dumbbells. The lounge has two sofas and a coffee table. The corridor has a console table with a vase on top.
Appendix Q User Study Interface Screenshots
Four complete figures preserve the study’s interactive viewer, fullscreen inspection, static alternatives, question/Equal-confirmation design and tutorial overlays. These screenshots document what participants could inspect and how responses were elicited.
Appendix R Additional Qualitative Results
The following examples illustrate connectivity, prompt-conditioned scenes, manipuland populations and themed generation. They are qualitative selections rather than additional independent success-rate estimates.
R.1 Baseline Comparisons
Figure 24 contrasts hotel connectivity, including Holodeck paths through guest rooms/bathrooms. Figures 25–26 compare house and room layouts. Figure 26’s caption says six baselines, but its visible rows contain five baseline configurations plus SceneSmith; the complete original art and caption are retained with this discrepancy noted.
R.2 Additional SceneSmith Examples
Figures 27–31 retain two multi-room and 24 room-level examples with their complete prompt annotations. They show diverse structures and dense contents, without establishing exhaustive compliance for every individual prompt.
R.3 Manipuland and Themed Generation
Figure 32 shows seven richly populated furniture/support examples whose small objects remain separable. Figure 33 shows six themed rooms with their prompts. Visual resemblance does not by itself verify material parameters or successful manipulation.
Appendix S Limitations and Failure Analysis
All 210 evaluation scenes were used without external discard or regeneration, but the pipeline internally retries assets and removes fallen objects. Remaining defects are documented explicitly; a locally imperfect scene can still be useful for simulation.
S.1 Representative Failure Modes
Figure 34 retains six failures: cabinet retrieved for a mini-fridge, opaque glass from missing PBR channels, incorrect wall-lamp orientation, cutlery not mirrored across a table, a TV fused into earlier-stage furniture, and a missed plant-stand support surface. These separate asset limitations, VLM geometry reasoning and irreversible stage decisions.
S.2 System Limitations
Cost and runtime favor high fidelity over minimum latency. Articulated diversity is limited by retrieval; static fallback cannot supply new mechanisms. Mass, friction and inertia are estimates rather than identified dynamics. Deformables, fluids, breakables and detailed material behavior are outside scope, and formal sim-to-real transfer remains future work.