Overview
This edition covers the complete twenty-one-page v2 dated 7 October 2026, including the additional r2Dreamer experiments. All nine figures, five numbered tables, the appendix contents table and seven numbered equations remain.
At a glance
World models can turn externally supplied demonstrations into synthetic training trajectories or learned simulation environments. The paper studies whether apparently safe demonstrations can carry image perturbations whose harmful effects emerge only during this transformation.
Visual Prompt Hijacking changes text-conditioned video generation; Visual Transition Hijacking changes action-conditioned predictions. The strongest downstream result is a backdoored reinforcement-learning policy evaluated in a simulator. The text-conditioned/VLA branch demonstrates corrupted video generation, but does not complete downstream VLA training.
1 Introduction
Robot policies benefit from large demonstration datasets, but collecting new scenes and embodiments is costly. World models promise reusable synthetic data and learned training environments, as in scalable world-model training and VLAW. Their position between third-party data and policy optimization creates a potential trust boundary.
The paper asks whether manipulating only the input images can poison that intermediate process. Its main findings concern a strong white-box threat model, increased prompt-hijacking success under camera/prompt shifts, and a downstream simulated RL backdoor. Claims of priority are the authors’ claims, not an independent survey conclusion.
2 Problem Setting
Both behavior cloning and reinforcement learning produce a policy from observation histories to action distributions in a partially observed decision process. The source defines a text-conditioned model as mapping a prompt and image to an observation/action trajectory, and an action-conditioned model as predicting the next observation.
In practice, text-to-video generation still needs a separate inverse-dynamics model to supply robot actions. That distinction matters because errors at this conversion stage prevent the paper from validating a complete VLA poisoning pipeline.
Equations 1
| (1) |
3 Related Work
The related work contrasts synthetic-data generation and learned simulators with earlier direct dataset or reinforcement-learning backdoor attacks.
World Models in Robot Learning
Text-conditioned video models such as Cosmos can propose demonstrations without an expert policy; an inverse-dynamics model then estimates action labels. Action-conditioned models such as DINO-WM support planning or learned simulation.
These routes reduce new data collection but inherit model errors and dependence on the supplied context. Teleoperation datasets, sim-to-real methods and open robot datasets supply the broader pipeline considered here.
Attacks against AI and Robotic
Earlier VLA backdoor attacks poison demonstrations directly, while reinforcement-learning backdoors may manipulate training rewards or transitions. This work instead alters images before a world model transforms them. The source’s broad claim that direct poisoning is trivial to detect is motivation, not a measured comparison in these experiments.
4 Identifying Vulnerabilities in the Robot Learning Supply Chain
A malicious data provider can alter demonstration video frames and has white-box access to the victim world model, including gradients. Actions, downstream prompts and reward functions are outside the attacker’s direct control. Perturbations are bounded in LAB color space; the intended constraint is that the original demonstrations remain safe and useful before synthetic generation.
This is stronger access than a black-box deployment threat. Publicly available model weights motivate the assumption but do not establish transfer to private models or unknown fine-tuning. Perceptual bounds are not a human imperceptibility study; the appendix explicitly calls the chosen budgets relatively large.
5 Methodology
A backdoor must change behavior when a trigger appears while preserving the intended task without it. Equation 2 formalizes the competing attack and clean-performance objectives. This separates targeted policy poisoning from indiscriminate degradation of the generated training data.
The two mechanisms exploit different model roles: text-conditioned generation can produce the wrong demonstration, while action-conditioned generation can make some actions appear preferable through distorted predicted outcomes.
Equation 2
| (2) |
5.1 Visual Prompt Hijacking Attacks
Visual Prompt Hijacking aligns the model’s internal semantic response under a benign prompt with a response associated with an adversarial objective. The paper implements this through the denoising vector field rather than readable instructions overlaid on the image. LAB-bounded perturbations preserve the nominal demonstration input while changing how the generator interprets it.
The prose compares the perturbed benign-prompt encoding with the original-image target-prompt encoding, whereas Equation 3 prints the perturbed image in both terms; the original expression is retained. The proposed link from corrupted videos to a downstream VLA backdoor is not experimentally completed here.
Equation 3
| (3) |
5.2 Visual Transition Hijacking Attacks
Unconditional transition hijacking pushes image embeddings away from ordinary model inputs, causing predicted trajectories to diverge from valid dynamics. That alone produces poor training data. The conditional objective instead preserves plausible prediction near a selected action while degrading alternatives, creating an indirect preference during policy learning.
Equations 4–5 retain the exact objectives. The claimed mechanism depends on the world model, reward construction and policy optimizer; prediction degradation alone is not equivalent to demonstrated policy compromise.
Equations 4–5
| (4) |
| (5) |
6 Experiments
Experiments evaluate the two branches separately. Text-conditioned Cosmos tests manually scored video behavior under prompt specificity and camera shifts. Action-conditioned DINO-WM supports complete PPO training and simulator evaluation; action-conditioned Cosmos tests only prediction changes.
Greater vulnerability under distribution shift is observed, but the proposed explanation in terms of uncertainty is not directly measured. Specific prompts and familiar cameras reduce many measured attack rates without constituting a certified defense.
6.1 Visual Prompt Hijacking Attacks
Fine-tune the two-billion-parameter Cosmos-Predict 2.5 model on LIBERO, then evaluate nine hazardous-object scenarios in three custom task families. Each object/setting uses five perturbed inputs and four generation seeds per input. Manual scoring follows whether the inferred robot motion would handle the original hazardous object, including cases where the video changes its appearance.
Table 1 shows strong heterogeneity. With the shifted camera and vague prompts, success ranges from 5% for oil to 100% for the gun; specific prompts lower many rates, but the gun remains at 95%. Under the familiar camera, most rates fall, although the vague-prompt gun case reaches 50% and the specific-prompt grater case remains 25%. Clean baselines are mostly zero, with some gun-scene failures.
Object size and learned task priors are proposed explanations, not isolated causal ablations. Collapsed or irrelevant videos count as attack failures under the targeted metric even when they are unusable training data. There are no reported confidence intervals or inter-annotator reliability estimates for this manual evaluation.
The attempted DreamGen-style pipeline fails to yield sufficiently reliable action-labeled demonstrations: generated physics and object identity can be inconsistent, and the fine-tuned inverse-dynamics model generalizes poorly to the new tasks. No downstream VLA poisoning result is established.
| Scene | Gift Box | Stove | Microwave | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Object | Bomb | Dynamite | Gun | Oil | Cardboard | Cereal Box | Fork | Knife | Grater |
| Out of Distribution Camera Position, Attack Success Rate | |||||||||
| Vague Prompt | 20% | 50% | 100% | 5% | 10% | 90% | 25% | 20% | 55% |
| Specific Prompt | 10% | 25% | 95% | 5% | 0% | 65% | 5% | 5% | 25% |
| No Attack | 0% | 0% | 10% | 0% | 0% | 0% | 0% | 0% | 0% |
| In Distribution Camera Position, Attack Success Rate | |||||||||
| Vague Prompt | 5% | 25% | 50% | 0% | 0% | 5% | 20% | 0% | 30% |
| Specific Prompt | 5% | 0% | 10% | 0% | 0% | 0% | 0% | 0% | 25% |
| No Attack | 0% | 0% | 5% | 0% | 0% | 0% | 0% | 0% | 0% |
6.2 Transition Hijacking Attacks
Train PPO inside DINO-WM’s Single Wall environment, poisoning 5% of initial states, and evaluate in the actual task simulator. Across five training seeds, poisoned policies move toward the adversary’s target action when triggered while retaining approximately baseline clean task performance. Figure 4 reports action proximity, cosine similarity and clean simulator success with standard-deviation bands.
Attack success is an action-space criterion, not a measured physical damage rate. The full pipeline is demonstrated on this simple simulated navigation task; Push-T training remains difficult.
For the two-billion-parameter action-conditioned Cosmos model, twenty attacks on DROID inputs reduce the difference between clean and perturbed predictions as actions approach the selected target. Figure 5 reports visual and embedding distances with standard deviations. This demonstrates action-dependent prediction distortion on real-world data, without training or deploying a downstream robot policy.
7 Conclusion
The results establish world-model inputs as a potential policy-poisoning entry point under white-box assumptions. More realistic environments, unknown models, physical robots, stronger perceptual constraints and complete VLA training remain open.
The authors propose post-generation consistency checks and robustness training of encoders/predictors. Their tested guardrails and automatic video labeling are inadequate in the reported settings, but this does not establish that every guardrail or current world model lacks protection. No evaluated general defense or robustness guarantee is provided.
8 Acknowledgments
The authors acknowledge support from DARPA, NSF, Amazon, Coefficient Giving, Schmidt Sciences and Nvidia. The source funding statements remain attributable to the authors.
Appendix Table of Contents
The original appendix contents table is retained. It omits the later B.1 and B.6 entries; both complete sections are included below.
| Section | Contents |
|---|---|
| Appendix A | Experimental Details |
| A.1 | VPH Attack Setup |
| A.2 | VTH Attack Setup |
| A.3 | Compute Resources Used |
| Appendix B | Additional Discussion and Experiments |
| B.2 | Further Analysis of Successful VPH Attack Instances |
| B.3 | Analysis of Prediction Sabotage in Text Conditioned World Models |
| B.4 | Preliminary Results Against World Model Guardrails |
| B.5 | Visualization of Visual Trajectory Hijacking Attacks |
Appendix A Experimental Details
The appendix specifies model adaptation, perturbation optimization, the learned-simulator reward and computing resources. These implementation assumptions bound the experimental conclusions.
A.1 VPH Attack Setup
LoRA adaptation uses the 256×256 LIBERO rerender associated with Cosmos Policy, while generating 432×432 videos. The authors report better video quality at higher resolution but choose this size for cost; the comparison is qualitative.
Table 2 preserves the exact projected-gradient/momentum parameters and LAB budgets. Optimization uses ten generated frames rather than the normal thirty-two to reduce cost. Step size is interpreted as a fraction of the per-channel bound, so its magnitude cannot be compared directly with an RGB attack budget.
Manual success labels track the original object and implied motion even when the generator changes a grater into food or a knife into another object. This procedure is necessary to understand the metric, but actual action reconstruction is not verified. The authors suspect single-view inverse dynamics contributes to the failed VLA pipeline; multi-view improvement is an untested explanation.
| Attack Parameters | ||||||
|---|---|---|---|---|---|---|
| Parameter | Num Steps | L | AB | Momentum | Frames Gen During Attack | |
| Value | 400 | 5 | 20 | 0.025 | 0.9 | 10 |
A.2 VTH Attack Setup
The action-conditioned experiments use base DINO-WM and Cosmos models. Optimization evaluates short target and non-target action sequences and averages the cosine objective over time, aiming for effects that persist near the chosen action rather than a single exact input.
A CleanRL PPO policy receives concatenated current and goal embeddings from a Gymnasium wrapper. DINO-WM’s frame skip means one model action corresponds to five repeated simulator actions.
Equation 6 combines an embedding-distance term with a goal bonus. The prose calls the first term a bonus for improvement, but the printed minimum with zero makes it nonpositive: it penalizes being farther than the best state and gives zero for improvement. The formula is preserved rather than silently changing its sign.
Equation 6
| (6) |
A.3 Compute Resources Used
A desktop with two RTX 4090 GPUs supports inference and DINO-WM experiments. Cosmos adaptation and perturbation optimization use a server with four H100 GPUs, reported as 96 GB each. Table 3 retains the complete CPU/RAM specification. Total GPU-hours, energy and per-attack runtime are not reported.
Appendix B Additional Discussion and Experiments
Additional analyses distinguish video-labeling errors, object-identity changes, non-targeted prediction collapse, guardrail observations and a modified attack on r2Dreamer.
B.1 Ineffectiveness of VLM Based Labeling
Qwen3-VL labels ten verified positive videos per object. Accuracy spans 10% for forks to 100% for oil and cardboard; it is 50% for graters. This small positive-only test supports concern about missed target actions, but does not estimate false positives or overall accuracy on a balanced success/failure set.
| Scene | Gift Box | Stove | Microwave | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Object | Bomb | Dynamite | Gun | Oil | Cardboard | Cereal Box | Fork | Knife | Grater |
| VLM Accuracy | 60% | 80% | 70% | 100% | 100% | 90% | 10% | 40% | 50% |
B.2 Further Analysis of Successful VPH Attack Instances
Successful examples often change the hazardous object into the prompt’s benign object while moving toward its original location. Figure 6 shows grater-to-steak and knife-to-cheese cases. This illustrates a mismatch between generated appearance and the original scene, rather than evidence that safe-looking video implies safe executable robot actions.
B.3 Analysis of Prediction Sabotage in Text Conditioned World Models
Other perturbations produce newly appearing objects or substitute familiar training objects, then fail both the adversary’s target and the requested task. Figure 7 retains these failure cases. Their potential harm to training is plausible, but a downstream degradation experiment for this subset is not reported.
B.4 Preliminary Results Against World Model Guardrails
The tested Cosmos guardrails do not block the authors’ hazardous direct prompts or perturbed-image demonstrations. The appendix gives observations rather than a detailed detection benchmark with sample counts, false-positive rates or guardrail-version comparisons. A proposed distinction between visual consistency filtering and robotics safety filtering is interpretation, not verified design documentation.
B.5 Visualization of Visual Trajectory Hijacking Attacks
Figure 8 compares clean and perturbed Cosmos predictions for target versus other actions; the non-target case shows more degradation. Figure 9 displays DINO-WM outcomes across a two-dimensional action grid and a clean reference rollout. These selected visual examples complement the quantitative metrics without measuring a physical attack.
B.6 Additional Results on r2Dreamer
The original transition-hijacking objective does not reliably collapse r2Dreamer: it often produces a believable but altered state. The authors therefore add a critic-based value term, explicitly manipulating whether alternative actions lead to lower-value predicted states. This requires additional critic access and differs materially from the main method.
Fresh PPO policies trained in this model use 25% poisoned states and are evaluated in Cartpole Swing Up and Reacher Easy simulators across five runs. Triggered action error drops from .49 to .08 and from 1.41 to .51; clean environment returns are 166 versus 256 and 626 versus 629. Table 5 retains the printed 95% intervals, including the unusually large .08±.5 entry.
The high poisoning fraction, two simple environments and modified objective limit generalization. These are supplementary evidence for the model-mediated poisoning route, not a replication of the lower-rate DINO-WM result.
| Env | Cartpole Swing Up | Reacher Easy | ||
|---|---|---|---|---|
| Policy | Clean | Poisoned | Clean | Poisoned |
| Triggered Action Error | 0.49 0.10 | 0.08 0.5 | 1.41 0.22 | 0.51 0.23 |
| Real Env Return | 166 86 | 256 37 | 626 12 | 629 28 |
Equation 7
| (7) |