Literature
Display
← All papers

Can MiniMax-H3 Reason About the Physical World? An Evaluation of Omni-Modal Generative Model

Haoyu Zhao · Zihao Zhao · Tianyu Deng · Ziqin Xu · Zihao Zhang · Xudong Wang · Jinxiang Guo · Chen Gao · Ziyi Ye · Yeying Jin · Jiaxi Gu · Zuxuan Wu · Shuicheng Yan

· arXiv (preprint) ↗

World models4 min text · CondensedOriginal paper ↗
Contents

At a glance

This study tests whether MiniMax-H3 can infer missing event information from complementary images, audio, and video. Human evaluation of 517 instances gives 41.97% overall success. Video-based decisions perform best (56.00%); audio-based disambiguation performs worst (27.40%). The benchmark tests the combined understanding-and-generation pipeline, so failures do not isolate reasoning from rendering errors.

1 Introduction

The benchmark deliberately withholds the answer from the text prompt. A correct generated video must recover it from the supplied observations. This probes cross-modal evidence use beyond reproducing an explicitly described scene. The dataset spans four reasoning scenarios and 29 subcategories; the results concern MiniMax-H3 rather than a comparison of competing models.

Related references: [21], [15], [37], [22], [26], [23], [12], [14].

Refer to caption
Figure 1: Overview of our evaluation for the reasoning capability of MiniMax-H3. First, we illustrate how MiniMax-H3 progresses from world observation and modality perception to omni-modal model training, where the acquired multimodal knowledge is transformed into task-oriented decision-making through omni-modal reasoning. Second, we construct implicitly paired multimodal data to systematically evaluate it across four complementary dimensions of physical-world reasoning.
Table 1: Comparison with existing evaluation benchmarks. Our evaluation extends beyond explicit text-image conditioning by introducing omni-modal inputs and implicit prompts, enabling the evaluation of multi-scene understanding, physical awareness, and multimodal reasoning.
Evaluation Sets # Examples Input Modalities Prompt Type Multi Scene Phys. Aaware Reasoning
VBench (Huang et al., 2024) 800 T + I Explicit ✗ ✗ ✗
TC-Bench (Feng et al., 2024) 150 T + I Explicit ✗ ✗ ✗
VideoPhy (Bansal et al., 2025) 688 T + I Explicit ✗ ✓ ✗
WorldModelBench (Li et al., 2026b) 350 T + I Explicit ✗ ✓ ✗
Ours 517 T + I + A + V Implicit ✓ ✓ ✓

2 Related Work

Prior benchmarks cover generation quality, physical consistency, and reasoning expressed through video.

Video generation evaluation.

Video benchmarks measure perceptual quality, temporal structure, and physical consistency. Here the task is to infer the event that should occur from incomplete multimodal evidence.

Related references: [12], [45], [48], [8], [46], [44], [47], [2], [3], [17], [43], [42], [24], [31], [5], [16], [13].

Reasoning through video generation.

Earlier work uses generated sequences as solutions to visual, symbolic, and planning tasks. This evaluation distributes the required evidence across views and modalities instead of specifying the solution in text.

Related references: [33], [30], [4], [19], [18], [40].

World-model evaluation.

World-model benchmarks test scene consistency, control, and state transitions. This study adds evidence-dependent generation: the observations determine the appropriate event or response.

Related references: [7], [14], [6], [35], [34].

3 Reasoning Evaluation with MiniMax-H3

The evaluation specifies four input configurations, constructs implicit prompts, and judges whether generated outcomes satisfy the corresponding semantic constraints.

Related references: [20], [11], [32].

3.1 What Do Omni-Models Enable?

Images supply layout, video supplies motion, and audio supplies event cues or spoken constraints. Their combination can make an otherwise ambiguous target inferable. Generated motion and state changes expose whether the model used that evidence.

Refer to caption
Figure 2: Video-audio generation pipeline of MiniMax-H3, which is an open-weight, general-purpose, omni-modal generation model.

3.2 Evaluation on Physical-World Reasoning Tasks

The four tasks assess cross-view geometry, audio-dependent interpretation, temporal decisions, and joint audiovisual reasoning. Text requests an operation while leaving the intended inference unspecified.

Equation 1
Y^∼pθ​(Y∣q,𝐱),\hat{Y}\sim p_{\theta}(Y\mid q,\mathbf{x}), (1)

Multi-view Spatial Reasoning (MSR).

MSR combines multiple views of one scene. Success requires consistent entity correspondence, occlusion handling, and spatial relationships that no single view fully reveals.

Equation 2
Y^MSR∼pθ​(Y∣q,𝐈).\hat{Y}_{\mathrm{MSR}}\sim p_{\theta}(Y\mid q,\mathbf{I}). (2)

Audio-based Disambiguation Reasoning (ADR).

ADR pairs an ambiguous image with a discriminating sound. The model must connect the sound to the correct object or event, such as identifying which differently made cup fell.

Equation 3
Y^ADR∼pθ​(Y∣q,I,A).\hat{Y}_{\mathrm{ADR}}\sim p_{\theta}(Y\mid q,I,A). (3)

Video-based Decision Reasoning (VDR).

VDR asks for a continuation responding to the dynamics in a video prefix. For example, a ball entering a road can justify stopping before a following child is visible.

Equation 4
Y^VDR∼pθ(Y∣q,V1:T),\hat{Y}_{\mathrm{VDR}}\sim p_{\theta}(Y\mid q,V_{1:T}), (4)

Audiovisual Integrated Reasoning (AVIR).

AVIR combines video with environmental audio or spoken constraints. The output must continue or correct the sequence using both sources; the audio need not be temporally aligned with the video.

Equation 5
Y^AVIR∼pθ​(Y∣q,V,A).\hat{Y}_{\mathrm{AVIR}}\sim p_{\theta}(Y\mid q,V,A). (5)
Refer to caption
Figure 3: Pipeline of evaluation data construction. We build implicit multimodal condition–prompt pairs in which the information required for correct generation is not explicitly stated in the prompt, but must be inferred from complementary visual and acoustic evidence.

3.3 Evaluation Data Construction and Human Annotation

Each example contains observations, an implicit prompt, and an annotated semantic target. Reviewers check that the target is inferable and can be assessed in a generated video.

Source Collection.

The authors combine real recordings and synthetic visual/audio sources, screening for perceptual quality and coherent task evidence.

Related references: [27], [29].

Task-Specific Instance Construction.

Inputs are arranged so the required spatial relation, sound source, anticipated response, or audiovisual correction depends on the observations. Success allows multiple valid appearances and motions.

Implicit Prompt Construction.

Language models propose prompt variants; experts edit them to avoid revealing the target or introducing unsupported assumptions.

Related references: [27], [36].

Iterative Expert Review.

Ten review rounds examine scene complexity, required inference, alignment of the observations, and prompt implicitness. Accepted examples cover 517 cases in four domains and 29 subdomains.

Original figure
Figure 4: Our evaluation set consists of 4 domains and 29 subdomains, totaling 517 paired conditions.

4 Experiments

Experiments measure overall success, differences across tasks, and recurring failures.

4.1 Evaluation Data and Protocol

The protocol uses task-specific human judgments rather than a single reference video.

Data sources and coverage.

The dataset has 200 MSR, 100 VDR, 146 ADR, and 71 AVIR instances. Sources include robot-manipulation views, general videos, sound datasets, and reviewed synthetic material. Scenario composition differs, which limits direct modality comparisons.

Related references: [1], [38], [25], [39], [10], [41], [9], [28].

Refer to caption
Figure 5: MSR evaluation inputs. Household and tabletop manipulation scenes include substantial viewpoint variation, occlusion, and diverse object configurations.
Refer to caption
Figure 6: ADR evaluation inputs. Scenes contain alternative candidate sound sources, including animals, instruments, tools, and appliances.
Refer to caption
Figure 7: VDR evaluation inputs. Selected video frames illustrate human and animal behavior, physical interactions, puzzles, and scene-memory settings.

Human evaluation.

Three experts independently assess outputs and cross-check their judgments. Realistic appearance alone is insufficient: the generated event must satisfy the evidence-dependent task requirement.

Metric and scope.

SR is the percentage of examples meeting the binary task criterion. Overall results are weighted by example count.

Equation 6
SR⁡(𝒟)=100|𝒟|​∑i∈𝒟si.\mathrm{SR}(\mathcal{D})=\frac{100}{|\mathcal{D}|}\sum_{i\in\mathcal{D}}s_{i}. (6)
Refer to caption
Figure 8: AVIR evaluation inputs. Selected frames span activities, animation, animals, and cues. Video is paired with environmental sounds or spoken constraints for continuation or localization.

4.2 Quantitative Results

The original table gives every subcategory count and success rate.

Table 2: Human-evaluated performance of MiniMax-H3 across four reasoning scenarios. Scenario-level and overall success rates are weighted by sample count. NN: number of samples. SR: success rate (↑\uparrow). Blank cells indicate no additional subcategories.
MSR VDR ADR AVIR
Subcategory NN SR (%) Subcategory NN SR (%) Subcategory NN SR (%) Subcategory NN SR (%)
Deformation 34 41.20 Humans 34 55.88 Vocalizations 16 31.30 Activities 14 57.14
Cleaning 27 33.30 Cartoons 5 20.00 Alerts 26 23.10 Animation 6 33.33
Articulation 26 34.60 Animals 11 63.64 Music 16 25.00 Making 21 52.38
Arrangement 23 47.80 Traffic 9 100.00 Machinery 34 17.60 Animals 10 30.00
Transport 23 34.80 Physics 10 50.00 Contact 32 21.90 Cues 20 50.00
Loading 20 50.00 Memory 6 66.67 Nature 22 54.50
Assembly 20 50.00 Puzzles 6 16.67
Pouring 12 58.30 Synthetic 19 52.63
Threading 8 75.00
Control 7 42.90
Overall 200 43.50 Overall 100 56.00 Overall 146 27.40 Overall 71 47.89

Overall success rate across all 517 samples: 41.97%.

Overall reliability remains limited.

Overall success is 41.97%. Scenario scores are 56.00% for VDR, 47.89% for AVIR, 43.50% for MSR, and 27.40% for ADR. The 28.60-point VDR–ADR gap reflects different tasks and datasets; it does not establish an isolated advantage of video over audio.

Results across scenarios.

Performance varies within scenarios: spatial threading reaches 75.00%, while several manipulation tasks remain near one third. Video puzzles reach 16.67%. Audio grounding remains difficult, especially for machinery, contact, and alerts.

Refer to caption
Figure 9: MSR results under paired-view conditioning. Each example shows the paired views together with the corresponding prompt.

4.3 Qualitative Analysis

Successful examples include coordinated manipulation stages, plausible short-term motion, and visible events guided by sounds. The figures show both inputs and generated frames.

Refer to caption
Figure 10: VDR results conditioned on prefix videos. Each row shows observed frames on the left and generated continuation frames on the right, together with the corresponding instruction.
Refer to caption
Figure 11: ADR results conditioned on visual and acoustic inputs. Each row shows the input scene, acoustic cue, and generated frames. The cabinet-opening and typewriter examples produce visible events consistent with the supplied sounds.
Refer to caption
Figure 12: AVIR results for discrepancy localization and audiovisual continuation. The upper examples localize visual content that conflicts with spoken constraints, while the lower examples continue the video from audiovisual inputs.

4.4 Failure Cases

Failures include binding evidence to the wrong object, omitting the required interaction, violating geometry or contact dynamics, and losing previously established state. Plausible-looking videos can therefore fail the actual reasoning task.

Refer to caption
Figure 13: Representative ADR failure cases. Each row shows uniformly sampled video frames and identifies the input sound and failure type: (a) audio-visual semantic mismatch, (b) implausible visual content, (c) confusion between similar sound sources, and (d) failure to ground a small-object interaction. Red boxes highlight the regions relevant to each failure.
Refer to caption
Figure 14: Representative VDR failure cases. Green-bordered frames indicate the observed prefix, followed by sampled continuation frames. Panels (a), (b), and (f) illustrate violations of physical constraints; (c) and (d) illustrate fine-grained manipulation errors; and (e) and (g) illustrate temporal inconsistencies in object states and scene content.

4.5 Discussion

The evaluation measures understanding, cross-modal integration, and generation together. Controlled removal or replacement of individual modalities would help distinguish their contributions and failure causes.

5 Conclusion

Implicit multimodal prompts expose a substantial gap between accepting different input types and using them reliably. Broader scenarios and reliable automated evaluation are proposed next steps.

Concept demonstration

Open image ↗