Literature
Display
← All papers

Training Object Permanence in World Models

Haotian Zhang · Fengyuan Yu · Dezhi Luo · Haoran Sun · Zehong Zhao · Qingying Gao · Yihan Li · Siyuan An · Huayi Qin · Yilan Zhang · Zhengze Jiang · Pinyuan Feng · Renrui Zhang · Ziyu Guo · Letian Wang · Mengyue Yang · Kangfu Mei · Maijunxian Wang · Ran Ji · Vikash Kumar · Freda Shi · Chandra Sripada · Vincent C. Müller · Philip Torr · Alan Yuille · Nikolaus Kriegeskorte · Felix Juefei-Xu · Lvmin Zhang · Jieneng Chen · Yilun Du · Hokin Deng

Preprint posted · arXiv (preprint) ↗

World Models10 min text · CondensedOriginal paper ↗
Contents

Overview

Project and released resources: WROP data, exam, model weights, code, answers, and leaderboard. The original figures and complete task inventory are retained below.

Refer to caption

Figure 1: WROP: World Reasoning with Object Permanence. We introduce a benchmark and training resource for object permanence and solidity in video generation models, built on 150 hand-designed Blender generators across six task families. Each generator randomises lighting, camera angle, speed, and other nuisance parameters while preserving the task’s core cognitive-scientific structure, yielding 10,000 samples per task. We release this 1.5M-sample training corpus, a 300-question evaluation exam with human Elo ratings across 14 video generation models, and PWM-WROP, a fine-tuned continuation model trained on WROP data achieving state-of-the-art performance in its category.

At a glance

Can video models keep track of objects when they disappear from view and respect solid boundaries? WROP supplies 150 hand-designed Blender generators across six task families, a 1.5-million-sample training set, and a 300-question exam. PWM-WROP, a 16-billion-parameter continuation model fine-tuned on this data, ranks first among four continuation systems and third across fourteen models in a blind human-preference study. The result supports targeted synthetic training, but different architectures and video interfaces prevent attributing the advantage to training alone.

1 Introduction

Realistic video can still violate object permanence (OP) and object solidity (OS): objects vanish during occlusion, change identity, or cross impenetrable barriers. Developmental studies treat these expectations as early foundations of physical cognition [2] [35]. WROP turns them into controlled three-dimensional video-continuation tasks, with ground-truth clips, prompts, object trajectories, and scene metadata.

The study combines a large training corpus with human judgments rather than relying on vision-language judges whose physical reasoning is itself unreliable [21]. PWM-WROP achieves Elo 1679.5 despite a native resolution of only 320 × 192. Two reference-to-video systems lead the overall ranking; their ability to regenerate an event on a new timeline makes their task different from faithfully extending the input.

2 Related Works

Video Models as World Models.

Diffusion and transformer-based generators have advanced visual fidelity, and benchmarks now probe maze solving, temporal induction, and visual rules [15] [30] [40]. WROP emphasizes a complementary gap: three-dimensional physical constraints, video-to-video conditioning, substantial training data, and human evaluation. Image similarity and automated language-model judgments can miss precisely the permanence and solidity failures under test.

Object Permanence and Solidity: Cognitive and Philosophical Foundations.

Piaget emphasized object permanence as a developmental achievement; later violation-of-expectation studies found that young infants already track hidden objects and expect containment and solid surfaces to constrain movement [31] [2] [13]. Adult visual cognition also exhibits automatic physical expectations [8]. These findings motivate the task design; success on the synthetic benchmark would not by itself establish a human-like internal representation.

3 Dataset

The dataset separates persistent object identity from the consequences of solid contact. All six families use a common input-video, prompt, and target-video format.

Refer to caption

Figure 2: Taxonomy of the six WROP task families. The top row evaluates object permanence through dynamic occlusion, static-scene occlusion, and container-based concealment and movement; the bottom row evaluates object solidity through obstruction, support removal, and collision. Each panel shows representative frames from a sample drawn from one generator in the corresponding task family: an early input frame, the final input frame at the split boundary, and a target frame depicting the expected physical outcome.

3.1 Cognitive Taxonomy

The 150 generators divide into 90 OP and 60 OS tasks. Structural variation changes the physical problem; surface variation changes appearance without changing the intended outcome. The event normally begins at the midpoint, after the conditioning clip.

  • OP-1, dynamic occlusion: a moving object must reappear with its identity, size, and direction preserved.
  • OP-2, static occlusion: a moving screen must reveal the same hidden objects in the same arrangement.
  • OP-3, containment: hidden contents must follow their container through displacement, rotation, or swaps.
  • OS-1, obstruction: aperture size determines whether a moving object passes or stops.
  • OS-2, support removal: an unsupported object must fall, with subsequent passage or support determined by geometry.
  • OS-3, collision: contact must produce plausible separate trajectories while conserving object identity and count.
Original figure
Figure 3: Generator-level composition of WROP’s 150 generators. The inner ring shows 90 OP and 60 OS generators; the outer ring shows the six task families.

3.2 Data Statistics

Each generator contributes 10,000 training samples and two exam questions: 1,500,000 training examples and 300 evaluation items. The standard sample is 120 frames at 1280 × 720 and 24 frames per second, divided into 60 conditioning and 60 target frames. It includes a prompt, per-frame object poses, and scene metadata. Motion is manually specified with Blender keyframes, rather than generated by a physics solver. The inference section separately identifies two 90-frame exam items.

3.3 Data Generation Pipeline

Generator design.

Each self-contained generator specifies objects, geometry, trajectories, contact timing, camera, prompt, and intended physical outcome. For example, marked boxes swap positions while retaining their original contents, requiring identity tracking rather than a fixed final-position rule. Authors inspect and revise scenes when collisions or occlusions look physically inconsistent.

A shared Blender 4.4.3 / EEVEE Next renderer records the random seed and packages input, target, prompt, trajectory, and metadata. Automated validation checks readable files, schemas, frame rate, clip lengths, and a continuous split; failures are regenerated. Representative samples from every generator receive manual inspection. Physical plausibility ultimately depends on the authored animation, not a rigid-body solver.

Refer to caption
Figure 4: Generator-design example from Marked_Boxes_Swap using the current 60-frame input and 60-frame target clips. The sequence shows the objects in marked boxes, the closed boxes after crossing at the split, completion of the position exchange, and the identity-preserving reveal. The task requires the model to maintain the association between each hidden object and its marked container across the input–target boundary rather than infer identity from final screen position.

4 Evaluation

Fourteen systems receive the conditioning video and unchanged prompt; the target remains hidden. Human pairwise preference is the primary measure. Full-reference video metrics provide secondary diagnostics of similarity to the target.

4.1 PWM-WROP

PWM-WROP fine-tunes Cosmos3-Nano for one epoch on 1.5 million earlier renders of the same 150 generators [28]. Architecture and tokenizer remain unchanged. Training uses only the clip and natural-language prompt, without task labels: 57 conditioning frames plus 60 predicted frames at 320 × 192. It is the only evaluated model trained on WROP.

The released PWM stack implements native PyTorch training on AWS Trainium2. A tensor-parallel × FSDP2 mesh spans 64 NeuronCores. At the separate 288 × 512 throughput-test geometry, engineering changes reduce batch-16 training steps from 15.1 to 5.71 seconds; these measurements are not the evaluation-video resolution.

4.2 Models Evaluated

Interface class changes what a system can do. True continuation extends the original clip: PWM-WROP, MAGI-1, LTX-2.3 Extend, and Grok Imagine. Reference-to-video regenerates an event from the source and prompt on its own timeline: Seedance 2.5, Wan 3.0 Prime, and MiniMax H3. Seven edit/transfer systems chiefly transform the source span, making events after the input boundary a poor match for their interface. Table 1 preserves access routes and output geometries; rankings should be interpreted within these differing contracts.

Model Access Interface class Native output (res / fps / frames)
Ours
PWM-WROP Trainium2 48XL True continuation 320×192320\times 192 / 24 / 60 predicted†
Open-weight
MAGI-1 24B Local, multi-GPU True continuation 1280×7201280\times 720 / 24 / 77
LTX-2.3 Dev Local, multi-GPU Edit / transfer (IC-LoRA) 1536×10241536\times 1024 / 24 / 121
Wan-VACE 14B Local, multi-GPU Edit / transfer (repaint) 832×480832\times 480 / 16 / 57
HY-OmniWeaving Local, multi-GPU Edit / transfer (editing) 848×480848\times 480 / 24 / 57
Cosmos3 Super Local, multi-GPU Edit / transfer (edge control) 1280×7201280\times 720 / 24 / 60
Proprietary
LTX-2.3 Extend FAL API True continuation 1920×10801920\times 1080 / 24 / 145‡
Grok Imagine (video extend) FAL API True continuation 1280×7201280\times 720 / 24 / 132‡
Seedance 2.5 FAL API Reference-to-video 1280×7201280\times 720 / 24 / 97
Wan 3.0 Prime FAL API Reference-to-video 1280×7201280\times 720 / 30 / 90
MiniMax H3 FAL API Reference-to-video 1344×7681344\times 768 / 24 / 124
Kling O3 Pro FAL API Edit / transfer (frame-wise) 1920×10801920\times 1080 / 24 / 73‡
Gemini Omni Flash 1.1 FAL API Edit / transfer (frame-wise) 1280×7201280\times 720 / 24 / 60
Runway Aleph 2 Runway API Edit / transfer (frame-wise) 1920×10801920\times 1080 / 24 / 60
Table 1: The fourteen evaluated models, grouped by provenance, with access route, interface class and native output geometry. †Our model emits a 117-frame clip that replays its 57 conditioning frames before the 60 predicted frames; the replay is trimmed before evaluation. ‡Stitched or padded outputs whose source prefix is trimmed before evaluation (Section 4.3).

4.3 Inference Protocol

The harness passes prompts verbatim, disables provider prompt expansion where possible, and validates returned videos. Open-weight systems use upstream-recommended settings; hosted requests adapt duration to each target. Short inputs are front-padded when required, and stitched source footage is removed before scoring.

PWM-WROP reads the last 57 input frames and predicts 60 with UniPC, 35 sampling steps, guidance 6.0, shift 10.0, and a fixed seed. For two 90-frame items it cannot see the first 33 conditioning frames. Provider-specific padding, trimming, request payloads, and returned geometry are logged; this normalization does not remove the fundamental interface differences.

4.4 Human Judgment

Twenty qualified raters see the input, prompt, and two anonymized continuations, then choose A, B, or a tie using prompt alignment, natural motion, and physical/object consistency jointly. Clips are normalized to silent 720p at 24 frames per second. Raters pass 90.0% of attention checks and agree with themselves on 93.5% of repeats; no significant left-position preference is detected.

The analysis uses 361 judgments between distinct models, about 50–52 per model. Bradley–Terry strengths are mapped to Elo with mean 1500; ties count as half-wins, with a small virtual-draw regularizer described in Table 2. Confidence intervals use 1,000 rater-clustered bootstrap resamples. These are pooled preferences over sampled comparisons, not a physical-correctness percentage across all 300 questions. The source also reports 476 completed judgments and 120 scheduled candidate pairs without fully reconciling that scheduling count with fourteen distinct models.

4.5 Automatic Metrics

Automatic measures include pixel error, PSNR, SSIM, MS-SSIM, LPIPS, temporal differences, final-frame metrics, CLIP similarity, and FID. Predicted clips are temporally resampled and generally compared at 320 × 192; additional metrics use their own specified resolutions. A static repaint can resemble the reference while failing to show the physical event, and a correct continuation can differ in timing. These scores measure target agreement, not physical understanding.

5 Results

5.1 Human Preference

5.1.1 Overall Results

Using the original Table 2 values, Wan 3.0 Prime and MiniMax H3 tie at Elo 1723.6; PWM-WROP is third at 1679.5 (95% interval 1603.5–1781.5), and Seedance 2.5 is fourth at 1649.6. PWM-WROP leads the next continuation model, Grok Imagine, by 222.5 Elo points. Wide overlapping intervals make adjacent ranks uncertain; bootstrap first-place frequencies are 46.8%, 36.4%, 12.3%, and 4.5% for these top four.

Reference-to-video systems can redraw a physically coherent event but may change identities or spatial layout. Editors cluster lower on this continuation task. PWM-WROP shows promising performance after targeted training, but different architectures and the lack of a matched untrained-base comparison prevent isolating a causal training effect. Some source prose values differ slightly from the retained numerical table; this summary follows the table.

Table 2: Human preference leaderboard of the 14 evaluated models on the 300-question WROP exam. Twenty raters made 361 blind pairwise judgments between the fourteen models (50–52 per model; ties count 0.5 for each side); strengths are Bradley–Terry maximum-likelihood estimates, with one virtual draw added per model and distributed evenly across its opponents, rescaled to an Elo scale with mean 1500. 95% confidence intervals come from 1,000 rater-clustered bootstrap resamples; overlapping intervals should not be read as significant rank differences. Score rate is the raw win rate (win =1=1, tie =0.5=0.5). Wan 3.0 Prime and MiniMax H3 have identical records and tie for first.
Rank Model Class Elo 95% CI Score rate Games
1 Wan 3.0 Prime Reference-to-video 1723.6 [1629.2, 1864.9] 77.9% 52
2 MiniMax H3 Reference-to-video 1723.6 [1644.5, 1837.6] 77.9% 52
3 PWM-WROP (ours) True continuation 1679.5 [1603.5, 1781.5] 73.1% 52
4 Seedance 2.5 Reference-to-video 1649.6 [1554.2, 1751.5] 69.6% 51
5 Runway Aleph 2 Edit / transfer 1518.3 [1425.1, 1621.6] 52.9% 51
6 Wan-VACE 14B Edit / transfer 1506.7 [1429.9, 1590.8] 51.0% 52
7 Gemini Omni Flash 1.1 Edit / transfer 1492.5 [1404.1, 1572.9] 49.0% 52
8 Kling O3 Pro Edit / transfer 1471.3 [1404.0, 1534.0] 46.2% 52
9 Grok Imagine (video extend) True continuation 1457.0 [1362.7, 1555.1] 44.2% 52
10 LTX-2.3 Extend True continuation 1453.4 [1363.6, 1545.8] 43.1% 51
11 Cosmos3 Super Edit / transfer 1409.2 [1318.2, 1488.7] 37.3% 51
12 LTX-2.3 Dev Edit / transfer 1398.9 [1297.7, 1482.3] 36.5% 52
13 HY-OmniWeaving Edit / transfer 1268.5 [1159.4, 1344.2] 21.0% 50
14 MAGI-1 24B True continuation 1248.0 [1137.2, 1315.6] 19.2% 52
Original figure
Figure 5: Human-preference Elo of the 14 models. Points are Bradley–Terry strengths on an Elo scale (mean 1500, dashed line), bars are 95% rater-clustered bootstrap intervals, colour is interface class, and PWM-WROP (filled) is the top true-continuation model.

5.1.2 Performance by Task Family

PWM-WROP ranks first on static-scene occlusion, third on dynamic occlusion and containment, second on support removal, fifth on obstruction, and eighth on collision. Its strength is therefore more consistent for hidden-object tracking than contact dynamics. MiniMax H3 leads support removal and collision; Wan 3.0 Prime leads containment. Per-family comparisons are sparse, especially for solidity tasks, so these ranks suggest patterns rather than establish statistically distinct abilities.

Refer to caption
Figure 6: Within-family human-preference ranks across the six WROP task families. Each cell shows the Bradley–Terry rank within that family (1 = best, 14 = worst), fitted separately on the 20-rater pairwise judgments for each family (36–96 games per family; 361 total). Color encodes rank from green (top) to red (bottom). The outlined row marks PWM-WROP; overall Elo (right) is from the joint fit over all judgments. Row order follows overall Elo rank; column groups correspond to the task families in Figure 2. Interface class is indicated by the label color, matching Figure 5.

5.2 Qualitative Analysis

Six selected examples illustrate the failure mechanisms rather than estimate their frequency. PWM-WROP preserves three balls through parallel tunnels and restores three objects after a moving screen; the shown Gemini and Seedance outputs instead add objects or relocate the hidden scene. In a rotating-cup example, PWM-WROP follows the original container while Seedance reveals the ball at its original screen position.

In the solidity examples, PWM-WROP stops a ball at an undersized aperture, makes it fall after support removal, and preserves seven balls through a pool break. Selected baselines pass through the barrier, delay the unsupported fall, or add/remove balls. The authors distinguish losing persistent object representations from generating events without their correct causal connections. These successful PWM-WROP examples coexist with its eighth-place aggregate collision ranking.

Refer to caption
Figure 7: Qualitative comparison on three object-permanence task families (Figure 2, top row). Each row shows the same ground-truth input (two key frames, blue), the PWM-WROP target output (two frames, green), and the full baseline output (three frames spanning the generation, red), with ✓\checkmark/×\times badges indicating physical correctness. G43 (OP-1): Gemini Omni Flash collapses all three balls into a tight cluster at the tunnel exit, violating their lane identities and count, while PWM-WROP keeps each ball in its correct lane with continuous motion. G19 (OP-2): Seedance 2.5 generates an oversized occluder over an empty region of the scene while the original objects remain visible, then makes all three objects suddenly appear when the occluder moves away, with no clear continuity from their previous positions; PWM-WROP correctly preserves the scene and reveals the same unchanged configuration. G66 (OP-3): Seedance 2.5 correctly animates the 180° turntable rotation but then lifts the cup at the original front position–a different physical cup–rather than tracking the cup containing the ball to its new location; PWM-WROP correctly tracks the ball with the rotating cup and lifts the correct cup.
Object-solidity examples comparing PWM-WROP with Seedance 2.5 and Gemini Omni Flash: blocking barriers, support removal, and count conservation through collision.
Figure 8: Qualitative comparison on three object-solidity task families (Figure 2, bottom row). Layout follows Figure 7: shared input (blue), PWM-WROP target output (green), and full baseline output (red). G10 (OS-1): Seedance 2.5 allows the ball to pass through the first barrier despite its aperture being half the ball diameter; PWM-WROP correctly brings the ball to rest at the barrier. G27 (OS-2): Seedance 2.5 removes the entire support structure rather than only the plate, then leaves the ball suspended in mid-air with no gravitational response; PWM-WROP generates an immediate, correctly-timed fall to the floor. G137 (OS-3): Gemini Omni Flash halluccinates additional balls before the collision, then allows the purple ball to vanish on impact, violating count conservation; PWM-WROP preserves all seven balls with physically plausible diverging trajectories.

5.3 Automatic Metrics

At 320 × 192, PWM-WROP has the best LPIPS (0.081), MS-SSIM (0.921), and MSE (0.00297) in Table 3. Other systems lead SSIM, PSNR, CLIP, or FID, confirming that no single similarity score captures physical correctness. Runway has 297 scored items because its API rejects three long prompts; other models have 300. Resolution changes and different output timelines further limit cross-model comparisons.

The table caption specifies CLIP at 224 × 224 and FID at 299 × 299, while the surrounding prose groups FID with 720p diagnostics. The original material is preserved; interpret the quantitative metric table using its stated evaluation settings.

Table 3: Automatic full-reference metrics against the reference continuation, averaged over the 300 exam questions and ordered by human Elo (Table 2). For each model, the predicted span is extracted using its per-model trim rule, uniformly resampled to the 60 target frames, and compared at 320×\times192; CLIP (ViT-B/32, 224×\times224) and FID (299×\times299, all frames at stride 2, ≈\approx9,030 frames per side) are computed at their respective input resolutions. The best result in each column is shown in bold and the second-best is underlined. These metrics measure similarity to the target rather than object-permanence capability: edit/transfer models can repaint the static scene and score well on structural metrics without depicting the required reappearance.
Model LPIPS↓\downarrow MS-SSIM↑\uparrow SSIM↑\uparrow PSNR↑\uparrow MSE (×10−3\times 10^{-3})↓\downarrow CLIP↑\uparrow FID↓\downarrow
Wan 3.0 Prime 0.115 0.861 0.942 27.15 4.48 0.948 15.1
MiniMax H3 0.105 0.877 0.938 27.51 9.09 0.962 14.8
PWM-WROP (ours) 0.081 0.921 0.917 26.45 2.97 0.956 20.4
Seedance 2.5 0.282 0.616 0.770 18.37 28.55 0.928 24.6
Runway Aleph 2† 0.181 0.789 0.918 24.98 9.71 0.918 19.7
Wan-VACE 14B 0.349 0.744 0.842 17.88 24.21 0.872 33.0
Gemini Omni Flash 1.1 0.151 0.830 0.937 26.81 3.88 0.943 17.0
Kling O3 Pro 0.163 0.818 0.933 26.10 4.73 0.932 18.0
Grok Imagine (video extend) 0.125 0.852 0.932 26.98 5.52 0.956 13.6
LTX-2.3 Extend 0.183 0.789 0.886 23.14 11.08 0.931 21.5
Cosmos3 Super 0.340 0.594 0.768 15.03 40.59 0.853 46.3
LTX-2.3 Dev 0.289 0.726 0.824 21.09 10.06 0.869 73.7
HY-OmniWeaving 0.247 0.716 0.874 22.41 17.61 0.875 43.8
MAGI-1 24B 0.260 0.681 0.853 21.59 20.66 0.923 26.6
† n=297n=297: the Runway API refused three prompts longer than 1,000 characters; all other models n=300n=300.

6 Conclusion

WROP offers controllable synthetic physical-reasoning data, a shared continuation exam, human judgments, and a reproducible model/training stack. PWM-WROP provides preliminary evidence that focused training can improve useful behavior, especially occlusion tracking.

Scope: training and evaluation use the same generator families, so this is not evidence of generalization to unseen physical tasks or real scenes. Hand-authored motion, unequal interfaces, absent matched-base ablations, and sparse human comparisons constrain broader claims. The work does not establish human-like cognition or reliable collision reasoning.

Acknowledgements

The authors acknowledge Amazon Web Services support through the AWS Trainium for Research program.

Appendix

Appendix A Task Inventory

The full 150-generator inventory is retained in Table 4, with task IDs, names, and physical scenarios for all six families.

Table 4: Task inventory of all 150 WROP generators across the six task families. Each entry names the generator and its core physical scenario.
ID Name Description
OP-1: Baillargeonian_Occlusion (26 tasks)
G02 ramp_tunnel An object rolls down a ramp, passes through an opaque tunnel, reappears …
G04 high_low_cover Objects pass behind high/low covers on crossing or straight paths; identity is preserved.
G14 u_tube_three_lanes Three coloured balls travel parallel U-tube lanes, preserve identity …
G17 moving_tray A moving tray carries objects behind a fixed screen (one or two lanes).
G28 open_ended_tunnel An object travels through an open-ended tunnel, hidden in the middle.
G29 container_entry_left_u One or two variably sized balls roll on two parallel left-U tracks.
G31 two_lane_tunnel Objects travel two-lane tunnels; identity is preserved.
G33 partial_window_parallel_cars Parallel wheeled cars pass behind a partial window.
G35 single_low_window An object passes behind a low window.
G36 rollercoaster_u_track_glassbox An object rolls a continuous U-track into a glass box.
G37 u_track_occluded_ball A ball travels a U-track behind an occluder.
G38 pendulum_occluded_by_screen A swinging pendulum is periodically occluded by a screen.
G39 lidded_box_ball_enters A ball enters a lidded box and is hidden.
G42 ring_track_behind_center_block A ball on a ring track passes behind a center block.
G43 three_balls_parallel_tunnels Three balls travel parallel tunnels; identity is preserved.
G54 serpentine_ramp_tunnel A ball rolls down a blue S-shaped serpentine ramp with three switchback segments …
G61 spiral_ramp_behind_column A ball rolls down a helical ramp that spirals around an opaque vertical column.
G62 two_balls_cross_tunnel A red ball on the left and a blue ball on the right roll toward each other and both …
G75 ball_behind_rotating_billboard A ball rolls from left to right at a steady speed while remaining in contact with the …
G76 pendulum_behind_post An opaque post stands in front of the lowest point of its arc …
G77 three_balls_one_wide_tunnel Three balls — red, green, blue, in that order — roll one behind another along a …
G78 rolling_disc_tunnel An upright disc (a coin standing on its edge) rolls along a straight flat track and …
G101 picket_fence_flicker A ball rolls at a steady speed along a straight rail behind a row of evenly spaced …
G102 drop_screen_occluder A ball rolls at a constant speed along a straight rail from left to right.
G103 corner_turn_occlusion A ball rolls along an L-shaped track.
G122 ball_behind_box_stack A ball rolls horizontally across the table at a steady speed and passes behind a tall …
OP-2: Object_Static_Occlusion (29 tasks)
G15 ramp_panel_occludes_objects Objects roll down a ramp behind a screen or panel and reappear.
G16 window_slit_mask_reveals A window or slit mask scans across the scene, revealing objects piece by piece.
G18 turntable_behind_screen An object on a rotating turntable is partially occluded by a fixed screen and must …
G19 row_screen_hides_objects A sliding foreground screen hides a row of objects.
G20 connected_vertical_panel A connected vertical guide panel hides objects behind it.
G21 connected_sliding_doors Connected sliding doors close over a center object and reopen.
G22 top_rail_screen A top-rail-mounted screen slides across to hide objects.
G23 cabinet A display cabinet’s cover or doors hide the objects inside.
G24 pivoting_occluder A pivoting sign or hinged page swings shut to hide objects.
G25 no_top_side_post_panel A side-post-mounted panel hides objects behind it.
G26 bottom_rail_screen A bottom-rail-mounted screen slides across to hide objects.
G47 theater_curtain A theater curtain closes over stage objects.
G48 sliding_window_row A sliding window reveals a row of objects.
G49 two_supported_screens_close Two supported screens close and reopen over objects.
G51 vertical_panel_two_objects A vertical rail panel hides two objects.
G53 no_gap_top_rail_curtain A no-gap top-rail curtain hides three balls.
G55 rising_floor_screen Two colored blocks rest side by side on a table.
G63 comb_occluder_sweep Three coloured blocks sit in a row on a table.
G64 flipboard_occluder A flat board, hinged along its bottom edge in front of them …
G79 venetian_blinds The slats all rotate to lie flat and fully occlude the objects, hold …
G80 sliding_double_doors Two opaque doors slide in from the left and right until they meet and fully cover the …
G81 rolling_shutter A segmented rolling shutter descends from above in front of them …
G82 accordion_fold_screen A folded accordion screen at one side unfolds sideways across in front of them until …
G104 bifold_concertina_doors A pair of bi-fold doors, hinged in the middle, unfold from the left and right until …
G105 descending_dome_cover A dome-shaped cover lowers straight down from above to fully enclose and hide them …
G106 rotating_drum_occluder The drum rotates about its vertical axis until its solid wall faces the camera …
G121 wiper_screen_occlusion A tall vertical screen, hinged at its base on one side …
G123 sliding_cover_panel A flat upright cover panel standing on a low track slides sideways in front of the …
G124 rising_sleeve_cover An open cylindrical sleeve, visibly wider than the pedestal …
OP-3: Container_Permanence (35 tasks)
G08 drawer_moves_hidden_object A closing drawer carries a hidden object as it slides.
G11 box_holds_N_objects A lidded box hides one to three objects through a rotation.
G12 partition_box A partitioned box hides several objects in a fixed order.
G13 three_drawers Three drawers hide objects in a fixed order.
G30 hidden_object_moves_with_cart A moving cart carries a hidden object.
G32 cart_swap Two carts swap positions while carrying hidden objects.
G40 guided_elevator_hidden_ball A guided elevator carries a hidden ball upward.
G45 opaque_lid_front_panel_drops An opaque lid or front panel drops over a center ball.
G46 marked_boxes_swap Marked boxes swap positions while hiding their objects.
G56 three_cup_shell_game Three identical opaque cups upside-down on a table …
G59 hinged_lid_box_opens A closed gray box whose top lid, hinged at the back edge …
G65 box_two_balls_relocate A lid closes over the box, hiding the balls; the box then slides across the table to a …
G66 rotating_carousel_cups The turntable rotates 180 degrees, carrying all three cups around with it.
G71 double_doors_swing_open The two doors swing open outward on their side hinges — the left door to the left …
G72 sliding_lid_box Its flat top lid slides off horizontally to one side …
G83 four_cup_three_swaps The cups slide through three sequential position swaps.
G84 two_balls_three_cups The cups slide through two position swaps.
G85 nested_cup_transfer A ball is shown, then a small opaque cup lowers over it.
G86 conveyor_covered_boxes Four identical opaque covers sit in a row on a conveyor belt …
G87 four_cup_carousel The turntable rotates 180 degrees, carrying all four cups around.
G88 shell_game_fakeout_reveal The cups slide through two position swaps.
G96 clamshell_box Its top half and bottom-front half swing open about a rear hinge — the top lifting up …
G97 rolltop_tambour A box with a curved roll-top (tambour) cover sits on a table.
G98 liftoff_dome_lid The dome lifts straight up, clearing the object, to reveal it resting on the base.
G107 sliding_cup_relocate A ball rests on a table; a single opaque cup is lowered over it …
G108 two_carts_cross_swap Two identical covered carts sit at opposite ends of a table …
G109 tilting_tray_relocate A lid closes over it; the tray then tilts so the hidden object slides under the cover …
G116 vault_swing_door A heavy round vault door on a side hinge is closed over the front of a safe.
G117 blooming_petal_box Four triangular flaps are folded up and inward to form a closed pyramid over an object.
G118 matchbox_drawer The inner tray then slides straight back INTO the sleeve horizontally …
G125 two_carts_reveal_empty Two identical covered carts sit at opposite ends of a table.
G126 turntable_two_boxes The front box’s cover lifts to show a ball under it, then lowers to hide it.
G131 stacked_drawers_reveal The lower drawer slides out toward the viewer, carrying the ball …
G132 twist_open_capsule The top half lifts straight up and off with a slight twist to reveal the ball resting …
G144 double_flap_top_box Starting open, both flaps swing down together and close over the ball …
OS-1: Baillargeonian_Obstruction (20 tasks)
G03 ramp_ball_blocked_by_wall An object rolls down a ramp into a solid or transparent wall and is blocked without …
G05 car_vs_barrier A vehicle meets a wall, hole, gate, or door — passing through an opening or being blocked.
G07 guillotine_gate_stops A vehicle or ball is stopped by a descending guillotine gate.
G09 low_beam_car_height A tall vehicle is blocked by a low beam while a short one passes underneath.
G10 size_gate A small object passes through an aperture; a larger one is blocked, rebounds slightly …
G34 occ_small_ball_behind_screen A small ball passes behind a foreground screen.
G50 ramp_ball_slotted_screen A ramp ball passes a slotted screen.
G52 ramp_open_right_gated_box A ball rolls down a right-opening ramp into a gated catch-box.
G57 ramp_ball_deflected_angled_wall A ball rolls down a ramp toward a fixed gray wall angled at 45 degrees and deflects …
G67 double_deflector_zigzag A ball is released and falls onto a left-leaning angled wall …
G68 width_slot_wall_two_balls A standing wall blocks a flat track, but it has a narrow vertical slot cut through its …
G89 limbo_height_bar A short object rolls along the track and passes cleanly UNDER the bar to the far side …
G90 funnel_size_sorter A small ball dropped into the funnel rolls down to the apex and passes through the …
G91 turnstile_timed_gate A ball rolls toward it and arrives when a gap between two arms is aligned with the …
G92 bumper_carom A ball rolls across a flat surface toward a fixed round bumper post (a cylinder).
G110 one_way_flap_gate A ball rolling in from the open side pushes the flap open and passes through …
G111 portcullis_drop_gate One ball rolls under it while it is up and passes through.
G112 banked_quarter_pipe_redirect A ball rolls toward a curved banked wall (a quarter-pipe).
G127 rising_bollard_stop A ball rolls along the table toward a spot at constant speed.
G128 swing_arm_barrier_stop A ball rolls up to the closed arm and is stopped – it decelerates to rest against the …
OS-2: Object_Drop (21 tasks)
G01 hole_box_drop An object drops onto a box with a covered or open center hole — it rests on the cap or …
G06 car_on_bridge A vehicle crosses a bridge that has a hole or a solid span; it falls through or passes.
G27 support_removed_then_fall A support is removed and the object falls.
G41 trapdoor_opens_ball_falls A trapdoor opens, the ball falls through to the floor …
G58 two_ball_trapdoor_size_filter A small ball and a large ball roll onto a gray platform with a circular hole in the …
G69 two_tier_trapdoor_cascade The trapdoor opens and the ball falls through onto a lower platform below …
G70 popaway_support_columns The two columns slide out sideways from under the slab …
G93 tipping_shelf_drop The prop slides out; with that end unsupported the shelf tips down …
G94 trapdoor_drop The trapdoor swings open downward on its hinge, removing the support …
G95 conveyor_edge_fall When it reaches the edge, it rolls off and falls into a bin (a lower catch floor) below.
G113 whipped_away_card The card is flicked out sideways fast; the ball, left unsupported …
G114 retractable_support_pins The two pins retract sideways into the wall, removing all support …
G115 steep_slope_slide It slides and rolls down the slope, accelerating under gravity, reaches the bottom …
G129 sliding_hatch_drop The hatch slides sideways fully clear of the hole, removing the support …
G130 bomb_bay_doors_drop The two doors swing downward and apart on their outer hinges (like bomb-bay doors) …
G145 tilt_platform_rolloff The platform tilts down on one side; the ball rolls to the low edge, rolls off …
G146 hanging_ball_release_drop The hook releases and the string goes slack, so the ball drops straight down to the …
G147 drop_leaf_shelf A ball rests on a shelf hinged at the back wall, held out horizontally like a …
G148 latch_release_flap_drop The pin is pulled out sideways; released, the flap flips down about its hinge and the …
G149 snap_pillar_topple Losing its support, the ball drops nearly straight down to the floor and bounces to rest.
G150 rollers_part_drop A ball rests nestled in the valley between two parallel horizontal rollers.
OS-3: Object_Collision (19 tasks)
G44 two_balls_collide_and_bounce Two balls collide and bounce apart.
G60 newtons_cradle The leftmost ball is raised and released; it strikes the row and the rightmost ball …
G73 break_scatter_cluster A cue ball rolls across a flat surface into a tight triangular cluster of stationary …
G74 glancing_oblique_collision A moving ball rolls across a flat surface and strikes a stationary ball off-centre (a …
G99 headon_velocity_exchange A ball rolls straight along a line into a second, identical stationary ball.
G100 heavy_light_collision A large heavy ball rolls into a small light stationary ball.
G119 offcenter_break_vsplit A ball rolls into the seam of two touching balls at rest, striking them off-center.
G120 pendulum_strike_projectile A heavy metal ball hangs as a pendulum, is raised and released …
G133 ball_topples_block A ball rolls across the flat floor and strikes a single standing block.
G134 knock_ball_off_tee A second ball rolls across the flat surface and strikes the resting ball.
G135 lever_launch_transfer A ball drops onto the raised end of a see-saw lever resting on a fulcrum.
G136 ball_strikes_pendulum A second ball rolls in along the table and strikes the hanging ball at the bottom of …
G137 pool_rack_break A ball rolls across a flat surface and strikes a triangular rack of six resting balls.
G138 bank_shot_carom A ball rolls across a flat surface, banks off a straight cushion at an angle (a clean …
G139 ball_shoves_block_slide A ball rolls across a flat surface toward a block resting upright on the table.
G140 light_ball_rebounds_off_heavy A small light ball rolls into a large heavy stationary ball.
G141 glancing_billiard_split A ball rolls across a flat surface and strikes a resting ball off-centre.
G142 knock_ball_off_ledge A ball rolls along the top of a raised platform toward a second ball resting at the …
G143 topple_two_blocks_apart Two tall blocks stand upright side by side on the table with a small gap between them.

Appendix B PWM on Trainium2

This appendix documents the native-PyTorch port, measured training throughput, and correctness checks for the released PWM stack.

Overview.

PWM loads public Cosmos3-Nano weights, shards them with DTensor over tensor parallelism and FSDP2, and compiles each transformer block once at fixed shapes. Training retains fp32 master weights with bf16 computation. Per-rank checkpoints support exact resumption and conversion back to the public layout. A YAML configuration controls geometry and parallel layout; invalid shapes, mesh sizes, precision, data, or resume settings fail before training.

Hardware.

A trn2.48xlarge supplies sixteen Trainium2 chips and 64 logical NeuronCores. The 36-layer model uses tensor-parallel degree four and FSDP degree sixteen. Inference needs only the four-core tensor-parallel group, with about 29 GB of bf16 weights, or 7.3 GB per core.

Throughput.

At 288 × 512, 117 video frames, batch sixteen, and 4,448 tokens per example, successive optimizations reduce step time from 15.1 to 6.74 to 5.71 seconds. Reported timings are medians of ten measured steps after three warm-ups; loss trajectories remain unchanged across configurations.

Table 5: Training step time on one trn2.48xlarge (36-layer Cosmos3-Nano, tp4×\timesfsdp16, batch 16, 288×512×30288\times 512\times 30 latent frames; medians over 10 steps after 3 warm-up steps). Loss trajectories are identical step for step across all rows.
Configuration s/step tokens/s
First working 64-core run 15.1 4,705
+ reduce-scatter copy-in rewritten (row concatenation) 6.74 10,564
+ multi-tensor AdamW, one sync per step; FSDP2 prefetch depth 2 5.71 12,454

Step time breakdown.

At the intermediate 6.74-second configuration, block forward/backward takes 3.4 seconds; communication, root-unit work, optimization, and clipping account for the remaining overlapping work. Collective latency dominates, so removing transfers helps more than shrinking their payload by reducing the FSDP degree.

Engineering findings.

  • Interleave loading and sharding to avoid memory fragmentation before the first backward pass.
  • Replace reduce-scatter chunk concatenation with row concatenation; the paper reports identical buffers and unchanged losses.
  • Batch AdamW updates with one synchronization per step, reducing optimizer time from 1,324 to 126 milliseconds.
  • The replicated vocabulary embedding still prevents a further memory optimization; vocabulary sharding or freezing the embedding remain proposed fixes.
  • An experimental fused attention backward path supports at most 8,192 tokens and has not received an end-to-end comparison; the compiled scaled-dot-product path remains the default.
  • Keep compiled blocks on the accelerator and pad/mask prompts to one static shape to avoid expensive host fallbacks and recompilation.

Correctness validation.

Validation includes bitwise CPU forward agreement, two-layer real-weight agreement within 7 × 10−7, exact 64-rank checkpoint resumption, and small-set overfitting under the continuation protocol. Generated-latent cosine similarity to an independent port is 0.88–0.97 across five same-seed prompts; numerical differences accumulate during 35 sampling steps. A complete 64-rank checkpoint occupies 162 GiB. These are implementation checks, separate from benchmark evidence about physical reasoning.

Appendix C Detailed Per-family Human Preference Data and Leaderboards

Table 6 retains every model’s family-specific Elo and rank, and Figure 9 shows bootstrap intervals. Solidity families have only 36–44 total comparison games each, leaving wide uncertainty. Prefer patterns repeated across families and supported by qualitative examples over small differences between neighboring ranks.

Table 6: Per-family Bradley–Terry Elo for all 14 models across the six WROP task families. Superscripts give within-family rank (1 = best). Underline marks the top-ranked model in each family; bold marks PWM-WROP (ours). Overall Elo is from the joint fit over all 361 judgments; models are ordered by overall Elo rank. Bootstrap CIs are wide especially in OS families (36–44 total games each); ranks indicate tendency rather than significant differences.
Model Class Overall Object Permanence Object Solidity
Elo OP-1 OP-2 OP-3 OS-1 OS-2 OS-3
Wan 3.0 Prime Ref.-to-video 1723.6 16702 15596 17301 15587 17493 19152
MiniMax H3 Ref.-to-video 1723.6 16144 16892 17072 16254 18681 19251
PWM-WROP (ours) True cont. 1679.5 16163 17851 16923 16045 18252 13948
Seedance 2.5 Ref.-to-video 1649.6 17101 16443 16564 16871 14897 17144
Runway Aleph 2 Edit/transfer 1518.3 144810 15527 15965 16413 14808 133112
Wan-VACE 14B Edit/transfer 1506.7 14867 143111 15787 16046 133812 14956
Gemini Omni Flash 1.1 Edit/transfer 1492.5 15236 145310 15826 124014 15254 138210
Kling O3 Pro Edit/transfer 1471.3 137313 15805 141210 15438 140510 17573
Grok Imagine (extend) True cont. 1457.0 14639 16014 14558 14799 120114 16335
LTX-2.3 Extend True cont. 1453.4 14838 15498 14529 130512 15006 132913
Cosmos3 Super Edit/transfer 1409.2 142611 14649 135411 16602 126313 13957
LTX-2.3 Dev Edit/transfer 1398.9 15805 141312 130012 143610 140311 133511
HY-OmniWeaving Edit/transfer 1268.5 137812 106614 122214 133811 15075 13839
MAGI-1 24B True cont. 1248.0 123114 121413 126413 128013 14489 101314
Original figure
Figure 9: Per-family human-preference Elo leaderboards for all six WROP task families. Points are Bradley–Terry MLE strengths on an Elo scale (mean 1500, dashed line); bars are 95% rater-clustered bootstrap intervals. Colour encodes interface class as in Figure 5: filled blue circle is PWM-WROP (ours); open circles are other true-continuation models (blue), reference-to-video models (orange), and edit/transfer models (green). Each panel is sorted independently by within-family rank. Intervals are especially wide in OS families (36–44 total games each); adjacent ranks are rarely distinguishable. Numerical values and within-family ranks are reported in Table 6.
Concept demonstration

Open image ↗