Literature
Display
← All papers

π₀.₇: a Steerable Generalist Robotic Foundation Model with Emergent Capabilities

Bo Ai · Ali Amin · Raichelle Aniceto · Ashwin Balakrishna · Greg Balke · Kevin Black · George Bokinsky · Shihao Cao · Thomas Charbonnier · Vedant Choudhary · Foster Collins · Ken Conley · Grace Connors · James Darpinian · Karan Dhabalia · Maitrayee Dhaka · Jared DiCarlo · Danny Driess · Michael Equi · Adnan Esmail · Yunhao Fang · Chelsea Finn · Catherine Glossop · Thomas Godden · Ivan Goryachev · Lachlan Groom · Haroun Habeeb · Hunter Hancock · Karol Hausman · Gashon Hussein · Victor Hwang · Brian Ichter · Connor Jacobsen · Szymon Jakubczak · Rowan Jen · Tim Jones · Gregg Kammerer · Ben Katz · Liyiming Ke · Mairbek Khadikov · Chandra Kuchi · Marinda Lamb · Devin LeBlanc · Brendon LeCount · Sergey Levine · Xinyu Li · Adrian Li-Bell · Vladislav Lialin · Zhonglin Liang · Wallace Lim · Yao Lu · Enyu Luo · Vishnu Mano · Nandan Marwaha · Aikys Mongush · Liam Murphy · Suraj Nair · Tyler Patterson · Karl Pertsch · Allen Z. Ren · Gavin Schelske · Charvi Sharma · Baifeng Shi · Lucy Xiaoyang Shi · Laura Smith · Jost Tobias Springenberg · Kyle Stachowicz · Will Stoeckle · Jiaming Tang · Jimmy Tanner · Shalom Tekeste · Marcel Torne · Kyle Vedder · Quan Vuong · Anna Walling · Haohuan Wang · Jason Wang · XuDong Wang · Chris Whalen · Samuel Whitmore · Blake Williams · Charles Xu · Sukwon Yoo · Lili Yu · Wuming Zhang · Zhuoyang Zhang · Ury Zhilinsky

Preprint posted · CoRL 2026 (accepted) ↗

Robotics11 min text · CondensedOriginal paper ↗
Contents

Overview

π₀.₇ combines a generalist action policy with detailed language, episode-quality metadata and optional generated image goals. This edition follows arXiv v2, including its complete 87-author byline, supplementary system details and evaluation rubrics.

Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 1: We introduce π0.7\pi_{0.7}, a steerable generalist robot foundation model that can perform dexterous tasks across many tasks, environments, and robots. π0.7\pi_{0.7} is trained with diverse prompts that contain not only the task description, but detailed language, generated subgoal images, and episode metadata. This provides richer context about not only what to do, but also how to do it, making it possible for π0.7\pi_{0.7} to leverage broad set of both robot and non-robot data and compose the skills in this data in new ways to solve new tasks.

At a glance

The central proposal is to label heterogeneous robot experience with context describing both the task and how it was performed. This lets one policy learn from demonstrations, imperfect autonomous episodes and non-robot data, then be prompted toward desirable behavior. Evaluations show strong performance on trained dexterous tasks, improved instruction following, transfer to new task–robot combinations and selected new tasks. Human coaching and separately trained high-level policies play an explicit role in the longer-horizon generalization results.

I Introduction

Simply pooling diverse trajectories risks mixing incompatible strategies and quality levels. π₀.₇ instead conditions on subtask descriptions, quality/speed/mistake labels and visual subgoals, with dropout allowing different prompt combinations at deployment. The claimed contribution is improved use of heterogeneous experience and evidence of skill recombination. The evaluation separates specialist-level performance on tasks present in training, instruction generalization in new environments, cross-embodiment transfer and new task compositions; these are different forms of generalization.

II Related Work

The model combines established components: a vision-language backbone, action generation, MEM history encoding, high-level language control and goal-image conditioning. Human video, web supervision and autonomous robot experience broaden its training mixture. The authors position detailed context conditioning as the main methodological contribution, rather than claiming invention of goal-conditioned control or world-model prompting.

III Flow-Based Vision-Language-Action Models

A VLA observes camera views and robot state, then predicts a short chunk of future actions through a smaller flow-matching action expert. Knowledge insulation trains the backbone with discrete FAST action tokens while blocking continuous-action gradients from entering it. Context ordinarily contains a task instruction; here it also disambiguates strategy and episode quality. The displayed likelihood objective is approximate because the flow-matching expert optimizes a bound rather than a closed-form action likelihood.

Equation 1
maxθ𝔼𝒟[logπθ(𝐚t:t+H∣𝐨t−T:t,𝒞t)].\max_{\theta}\;\mathbb{E}_{\mathcal{D}}\left[\log\pi_{\theta}(\mathbf{a}_{t:t+H}\mid\mathbf{o}_{t-T:t},\mathcal{C}_{t})\right]. (1)

IV π 0.7 Overview

The action model has approximately 5B parameters: a Gemma3 4B backbone, including its 400M vision encoder, plus an 860M action expert. The memory encoder compresses multiple observation frames into a fixed token budget. A separate high-level policy can supply semantic subtasks, and a separate BAGEL-derived world model can supply image goals. Their parameters and compute should not be conflated with the 5B action model alone.

Original figure
Fig. 2: Architecture overview. The π0.7\pi_{0.7} model is a 5B-parameter VLA consisting of a 4B VLM backbone, a MEM-style video history encoder and a 860M parameter action expert. The model’s context includes multiple distinct modalities, including language commands, episode metadata that describes the data quality and strategy, and multimodal inputs such as subgoal images. At runtime, the language commands are produced by a high-level semantic policy based on the same architecture, and the subgoal images are produced by a lightweight world model based on the BAGEL image generation model [105].

V Diversifying the Prompt

Prompt components carry complementary information: language describes the intended operation, images depict a desired state, and metadata identifies the quality or execution mode represented by an episode. Component dropout makes these inputs optional at test time.

V-A Subtask instructions

Subtask text specifies the next semantic step in addition to the overall goal. It can come from a trained high-level policy, live human coaching, or be omitted. Coaching a new task supplies timed language instructions without new teleoperated action demonstrations. Turning those episodes into an autonomous system then requires fine-tuning a high-level language policy to produce the instructions from observations and history.

V-B Subgoal images

Multi-view goal images specify near-future scene and gripper configurations that can be difficult to express in words. A separate world model receives current views, subtask language and metadata; it is initialized from BAGEL, a 14B multimodal model, and trained to predict segment-end views. Robot, human-video, image-editing and other video data supply semantic and physical context. Better performance with goal images can therefore reflect additional world-model knowledge and compute, as well as a richer control prompt.

Equation
maxψ⁡𝔼𝒟g​[ℒCFM​(𝐠t⋆,gψ​(𝐨t,ℓ^t,m))],\max_{\psi}\;\mathbb{E}_{\mathcal{D}_{g}}\left[\mathcal{L}_{\text{CFM}}\left(\mathbf{g}_{t}^{\star},\;g_{\psi}(\mathbf{o}_{t},\hat{\ell}_{t},m)\right)\right],
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 3: Prompt overview. π0.7\pi_{0.7} uses diverse modalities of context in the prompt, including: subtask instructions, subgoal images, and episode metadata. We train the model with dropout for each component, and then prompt the model flexibly combining modalities. For example, when using the UR5e bimanual manipulator to fold a shirt, we use subgoal image and metadata prompting.

V-C Episode metadata

Episode metadata includes length discretized in 500-step bins, human-rated quality from one to five, and segment-level mistake labels. These labels tell the policy which behavior mode a training example represents, allowing lower-quality and failed episodes to remain useful. At deployment, requesting high quality and no mistakes steers the conditional policy; those labels describe the desired mode rather than guaranteeing an error-free execution.

V-D Control mode

A textual control-mode identifier selects joint-space or end-effector actions. Both appear during training, and the identifier is always present. Runtime selection changes the action representation used by the controller.

V-E Full prompt and training details

Only 25% of training examples receive image goals, because those examples make action prediction an easier inverse-dynamics problem. Among them, subtask text is omitted 30% of the time. All metadata is dropped 15% of the time, with an additional independent 5% dropout per metadata component. Control mode is never dropped. The original example prompt is retained as a diagram.

Complete original prompt: multi-view observation and subgoals; task peel vegetables; subtask pick up the peeler; speed 8000; quality 5; mistake false; joint control; proprioception.

VI The π 0.7 Model and Training Recipe

The recipe combines heterogeneous data, memory-aware observation encoding and goal-image training that anticipates imperfect runtime generations.

VI-A Training datasets

Training mixes demonstrations across robot types and environments, autonomous policy evaluations, human interventions, public robot data, egocentric human video and web-based vision/language supervision. It also includes failed or imperfect episodes and rollouts from earlier RL specialists, effectively distilling their behavior. The authors explicitly exclude autonomous data from generalization-focused evaluation tasks. Thus “out of the box” on trained tasks means no task-specific post-training of π₀.₇, not absence of task experience or specialist-generated training data.

VI-B Model architecture

Inputs include up to four camera views with six frames each, sampled one second apart, and up to three goal views. Images are resized to 448×448; history and the optional rear view each have 0.3 dropout. Proprioception uses learned linear projections, with dropped history states masked consistently. Block-causal attention separates observation, goals, text and action groups. The expert generates 50-step chunks, while training-time real-time chunking simulates delays of up to twelve steps, equivalent to 240 ms at 50 Hz.

VI-C Training with subgoal images

For real-image goals, a quarter of examples use the end of the annotated segment and the rest use a future frame sampled zero to four seconds ahead. Additional examples use goals generated by the world model. This exposure reduces the mismatch between clean future observations during training and imperfect or delayed synthesized goals during deployment.

VII Prompting π 0.7 at runtime

Runtime prompts always include control mode and metadata: quality five, no mistake, and a task-specific speed target based on the 15th percentile of episode length. Subtasks come from coaching or the high-level policy. Optional image goals refresh when the subtask changes or four seconds elapse, with generation running asynchronously. The action expert uses five denoising steps and executes fifteen or twenty-five steps of each fifty-step chunk. Metadata classifier-free guidance uses weights 1.3, 1.7 or 2.2 for selected dexterous tasks; the original workflow and guidance equation are retained.

Equation
∇𝐚\displaystyle\nabla_{\mathbf{a}} logπθ(𝐚t:t+H|𝐨t,𝒞t)+\displaystyle\log\pi_{\theta}(\mathbf{a}_{t:t+H}|\mathbf{o}_{t},\mathcal{C}_{t})+
β\displaystyle\beta (∇𝐚logπθ(𝐚t:t+H|𝐨t,𝒞t)−∇𝐚logπθ(𝐚t:t+H|𝐨t,𝒞tuncond)),\displaystyle(\nabla_{\mathbf{a}}\log\pi_{\theta}(\mathbf{a}_{t:t+H}|\mathbf{o}_{t},\mathcal{C}_{t})-\nabla_{\mathbf{a}}\log\pi_{\theta}(\mathbf{a}_{t:t+H}|\mathbf{o}_{t},\mathcal{C}_{t}^{\text{uncond}})),
Algorithm 1 Prompting π0.7\pi_{0.7} at test time
1: Input: initial observation 𝐨0\mathbf{o}_{0}, task instruction ℓ\ell, episode metadata mm, control mode cc
2: Initialize subtask ℓ^\hat{\ell} (from high-level policy or coaching)
3: 𝐠⋆∼pψ​(𝐠⋆∣𝐨0,ℓ^,m)\mathbf{g}^{\star}\sim p_{\psi}(\mathbf{g}^{\star}\mid\mathbf{o}_{0},\hat{\ell},m)
4: 𝒞={ℓ,ℓ^,𝐠⋆,m,c}\mathcal{C}=\{\ell,\hat{\ell},\mathbf{g}^{\star},m,c\}
5: 𝐚t:t+H∼πθ(𝐚∣𝐨t−T:t,𝒞)\mathbf{a}_{t:t+H}\sim\pi_{\theta}(\mathbf{a}\mid\mathbf{o}_{t-T:t},\mathcal{C}) ⊳\triangleright Optional: CFG
6: for t=0,1,2,…t=0,1,2,\ldots do
7:   if ℓ^\hat{\ell} changed or Δ\Delta-second timer elapsed then
8:    𝐠⋆∼pψ​(𝐠⋆∣𝐨t,ℓ^,m)\mathbf{g}^{\star}\sim p_{\psi}(\mathbf{g}^{\star}\mid\mathbf{o}_{t},\hat{\ell},m) ⊳\triangleright Non-blocking (async)
9:    𝒞={ℓ,ℓ^,𝐠⋆,m,c}\mathcal{C}=\{\ell,\hat{\ell},\mathbf{g}^{\star},m,c\}
10:   end if
11:   if H^\hat{H} steps elapsed since last inference then
12:    𝐚t:t+H∼πθ(𝐚∣𝐨t−T:t,𝒞,𝐚t:)\mathbf{a}_{t:t+H}\sim\pi_{\theta}(\mathbf{a}\mid\mathbf{o}_{t-T:t},\mathcal{C},\mathbf{a}_{t:}) ⊳\triangleright Async w/ RTC
13:   end if
14:   Execute 𝐚t\mathbf{a}_{t}
15: end for

VIII Robot system details

Experiments use mobile bimanual robots, lightweight static bimanual robots, single-arm systems and a larger UR5e platform with different reach, inertia and grippers. All use parallel-jaw grippers and front/wrist cameras; mobile robots add a rear camera. UR5e runs at 20 Hz and the other platforms at 50 Hz. Joint targets use a PD controller; end-effector targets are converted through numerical inverse kinematics. Cross-embodiment evaluation transfers tasks to robots without examples of those tasks, rather than claiming the target robot itself is entirely unseen.

Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 4: Illustrations of some of the robots in our experiments. We evaluate π0.7\pi_{0.7} on a variety of robots, including bimanual mobile manipulators (left), static bimanual robots (middle), and a bimanual UR5e setup (right) that we use for cross-embodiment experiments.
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 5: Illustration of selected evaluation tasks. We evaluate π0.7\pi_{0.7} on a number of tasks, and two of the more longer-horizon ones are visualized here. For some tasks such as “Take Out Trash”, we provide a coarse instruction like “take out the trash” and π0.7\pi_{0.7} performs the full long-horizon task. For other tasks which do not appear in the training data for π0.7\pi_{0.7} such as “Toasting a Bagel”, we can leverage the strong language following capabilities of π0.7\pi_{0.7} to coach it to perform the task with a series of detailed instructions that break down the task step-by-step.
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 6: Out-of-the-box dexterity: π0.7\pi_{0.7} can perform a wide range of highly dexterous tasks directly out of the box. We consider tasks from π0.6∗\pi^{*}_{0.6} [50] (top row) and a number of other dexterous tasks including ones from the “Robot Olympics” experiments (bottom row). For the tasks from π0.6∗\pi^{*}_{0.6}, we report success rate and normalized throughput (relative to the specialist model; raw throughput means successes per hour), while for other tasks we report task progress. We find that the same π0.7\pi_{0.7} model can match the performance of the task-specific post-trained specialist policy from π0.6∗\pi^{*}_{0.6} or π0.6\pi_{0.6} for each of these tasks, and even achieve higher throughput than the RL specialists in diverse laundry folding and box building.
Original figure
Fig. 7: Impact of prompt composition and evaluation data on out-of-the-box performance: We compare π0.7\pi_{0.7} with two ablations: one that does not include episode metadata in the context, π0.7\pi_{0.7} (no metadata), and another that does not include data from autonomous evaluation episodes during training, π0.7\pi_{0.7} (no eval data). We find that π0.7\pi_{0.7} outperforms both π0.7\pi_{0.7} (no metadata) and π0.7\pi_{0.7} (no eval data) across the board, with the gap most prominent in throughput. Throughput (successes/hour) here is normalized relative to π0.7\pi_{0.7}.
Original figure
Fig. 8: Tasks that require memory: π0.7\pi_{0.7} can also perform tasks that require explicitly keeping track of prior context, achieving similar or better performance compared to the specialist policies with memory fine-tuned to some of the tasks in the MEM paper [37].
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 9: Broad instruction following in novel environments: We evaluate π0.7\pi_{0.7} on 14 instruction following scenarios, each of which involve following a sequence of 3-6 open-ended instructions, across 4 unseen kitchen and 2 unseen bedroom environments. We report the instruction following success rate, the percentage of total instructions that were correctly followed across all evaluations. We find that π0.7\pi_{0.7} significantly outperforms π0.5\pi_{0.5} and π0.6\pi_{0.6} across the board, attaining high absolute success rates.
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 10: Following complex referential instructions: π0.7\pi_{0.7} and prior models all succeed on the simpler re-arrangement instructions (instructions include “pick up the spoon”, “put the spoon to the left of the fork” and “put the spoon to the right of the fork”), but π0.7\pi_{0.7} performs significantly better on the complex and unusual instructions (instructions include “pick up the largest bowl on the table”, “pick up the object I would use to eat soup” and “pick up the fruit on the largest plate”). Including subgoal images generated by a lightweight world model (π0.7\pi_{0.7} (GC)) further boosts instruction following performance, making π0.7\pi_{0.7} significantly more capable at following complex instructions.
Original figure
Fig. 11: Breaking dataset biases by following instructions: the improved language-following performance of π0.7\pi_{0.7} enables it to break strong dataset biases. Prior models struggle on these data bias challenge tasks, which require following instructions that contradict the pattern in the data (e.g., putting dishes in the trash and trash in the dish bin). π0.7\pi_{0.7}, however, can follow instructions sufficiently well that it can break these strong biases to still perform the task. Notably, for the “Reverse Fridge to Microwave” task, including subgoal images from the world model in the context (π0.7\pi_{0.7}(GC)) is critical for success.

IX Experimental Evaluation

The evaluation tests five questions: performance on trained dexterous tasks, instruction following, transfer across embodiments, composition of unfamiliar tasks, and how context changes the value of mixed-quality or diverse training data. Figures use several distinct metrics, including binary success, partial task progress and normalized successes per hour.

IX-A Out-of-the-box performance on challenging tasks

One π₀.₇ policy approximately matches task-specialized RL or supervised policies on trained tasks such as espresso preparation, laundry and box building, and can exceed specialist throughput on some tasks. Other evaluations include sandwich preparation, shirt inversion, door traversal, cutting, peeling and trash replacement. Removing episode metadata or autonomous evaluation data lowers performance, especially throughput, supporting the value of contextualized specialist experience. Memory tasks include mug sequencing, finding a hidden object, counting scoops and window cleaning; the generalist is comparable to memory-equipped specialists. These results are not zero-shot task learning.

IX-B Instruction following

Four unseen kitchens and two unseen bedrooms provide fourteen scenarios with three-to-six instructed steps each; the metric is the fraction of individual instructions followed, not whole-sequence success. π₀.₇ improves over π₀.₅/π₀.₆, especially for unusual referential commands, with further gains from generated goals. Reverse-bussing and microwave-to-fridge tasks deliberately contradict common training patterns. Goal images are particularly important for the reversed appliance sequence. Novelty is controlled by scenario design but cannot be exhaustively established against the entire large training mixture.

Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 12: Cross-embodiment transfer: Left: Both π0.7\pi_{0.7} and prior models achieve strong cross-embodiment transfer directly out of the box on simpler re-arrangement or repositioning style tasks. For example, for the “Table Setting” task, data was collected with a variety of different robots, and the task was tested on the static bimanual robot. All of the models performed well. For tasks that required transfer from the bimanual UR5e robot to the smaller static bimanual robot (“Bag In Backpack” and “Organize Tupperware”), the embodiment gap is larger, as there is only one source robot that is bigger and heavier than the target robot. Here, π0.5\pi_{0.5} performed very poorly, but π0.6\pi_{0.6} still performs quite well. Transferring from the smaller static bimanual robot to the UR5e for the “Shirt Bagging” task introduces the largest embodiment gap, and here the π0.7\pi_{0.7} model significantly outperforms prior models. Right: for the more dexterous tasks that require folding towels and t-shirts, the embodiment gap poses an even greater challenge. Data for these tasks was collected with the smaller static bimanual robot, and the task was evaluated on the larger bimanual UR5e platform. π0.7\pi_{0.7} was able to transfer this task successfully, and improved even more when using visual subgoal images generated by our lightweight world model. In fact, task progress matches the “zero-shot” performance of our most experienced human teleoperators, who have operated a diverse set of robots and attempted this task on the UR5e for the first time.

IX-C Cross-embodiment transfer

Older models also transfer simpler rearrangement tasks, while π₀.₇ has larger gains for demanding embodiment shifts. Examples include a single UR5e arm bagging shirts and two UR5e arms folding cloth despite no UR5e laundry demonstrations. Qualitative rollouts show different grasp and insertion strategies suited to the target geometry. The goal-conditioned model reaches 85.6% shirt-folding progress and 80% success; the reported expert-human comparison is 90.9% and 80.6%, respectively. This compares experienced operators making their initial attempts on this task–robot combination, not unrestricted human performance. Joint-space control is used in the main comparisons.

Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 13: Cross-embodiment transfer produces emergent strategies adapted to the target embodiment. (a) On the source robot, human teleoperators use one arm to hold the bag open while the other performs insertion. On the UR5e target robot, π0.7\pi_{0.7} instead discovers a single-arm pick-and-place strategy suited to the robot’s greater reach. (b) Human teleoperators approach the shirt with a tilted end-effector on the source robot, while π0.7\pi_{0.7} produces vertical grasps on the UR5e, which is more suitable for the larger robot’s arm placement. In both cases, the policy goes beyond replicating source behavior, discovering manipulation strategies for the task that are better suited to the target embodiment.
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 14: Example of language coaching: We can “teach” a new task to π0.7\pi_{0.7} by providing step-by-step verbal instructions. Because of its language following ability, π0.7\pi_{0.7} can perform new tasks successfully under user instruction, and these instructions can then by used to train a high-level policy that prompts π0.7\pi_{0.7} so that it can perform the task fully autonomously.
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 15: Coaching to perform new long-horizon tasks: Because π0.7\pi_{0.7} can follow language instructions effectively, even for unfamiliar skills, it can be “coached” to perform a number of unseen, longer horizon tasks both when conditioned on language and generated subgoal images (π0.7\pi_{0.7} (GC)). Prior models generally lack the language following ability needed to follow the coaching commands, and thus achieve very poor performance.
Original figure
Fig. 16: Acquiring new autonomous capabilities with coaching: We can use the coaching episodes collected for a number of different unseen tasks to train a high-level policy to automatically prompt π0.7\pi_{0.7} in accordance with the coaching episodes. This allows us to create fully autonomous policies for these tasks (π0.7\pi_{0.7} (autonomous)) that closely match the performance of the policy with live human coaching (π0.7\pi_{0.7} (coaching)) without collecting any additional data with teleoperation or any other kind of low-level actions.
Original figure
Fig. 17: Performing new short-horizon tasks: π0.7\pi_{0.7} can perform a number of new short-horizon tasks directly out of the box, including scooping rice into a rice cooker, spinning various objects such as a gear set and desk fan, and wiping down objects with a cloth, such as a ruler and headphones, despite no data being collected for any of these tasks. π0.7\pi_{0.7} shows roughly equally strong performance when conditioned directly on language instructions (π0.7\pi_{0.7}) or generated image goals (π0.7\pi_{0.7} (GC)).
Original PDF visual retaining vector text, complete panel arrangement and native embedded image detail.
Fig. 18: Scaling of generalization performance with diverse context and data: Left: We find that π0.7\pi_{0.7} (with metadata) can continuously improve its performance when it is trained on larger datasets, even when the average quality of the data actually decreases. By contrast, without training on rich conditioning information, we find that π0.7\pi_{0.7} (without metadata) actually can degrade in performance as more lower quality data is introduced. Right: When π0.7\pi_{0.7} is trained without our robot data with the highest task diversity, its performance degrades substantially. This suggests that π0.7\pi_{0.7} is able to utilize the task diversity in our robot data to drive substantial improvements in compositional task generalization.

IX-D Compositional task generalization

Selected new short tasks include pressing a French press, scooping rice, wiping objects and spinning articulated items. Longer appliance tasks require step-by-step human coaching: loading/unloading an air fryer and toasting a bagel. No targeted robot action data were collected for those tasks, although related appliances appear in human and external data. Recorded coaching episodes then train a high-level instruction policy; its autonomous performance is close to coached performance on five tested tasks. This removes the need for additional teleoperation demonstrations but still uses collected coaching data and high-level training.

IX-E Can π 0.7 learn effectively from diverse and mixed-quality data?

Eight laundry models compare four nested data subsets—the top 30%, 50%, 80% and all episodes by quality/speed—with and without metadata. More mixed-quality data improves the metadata-conditioned models, whereas the unconditioned versions can degrade. A second ablation removes either the most task-diverse 20% or a random 20%; removing diverse data harms short-task generalization more at matched quantity. The results support complementary roles for context and diversity, while the authors acknowledge that slicing diversity cleanly remains difficult.

X Discussion

Detailed context makes heterogeneous experience more useful and allows a single action policy to express multiple strategies. Generalization remains weaker than trained-task performance: the discussion describes unseen task or task–robot success around 60–80%, versus often above 90% on seen tasks. Related behaviors may occur incidentally somewhere in the training mixture, so “unseen” means no deliberate targeted collection in the relevant evaluation, not proven absence of every constituent skill. Future work includes improving new-task reliability through coaching or autonomous learning; the reported system already relies on substantial annotation and auxiliary-model compute.

Acknowledgements

The authors acknowledge the operators responsible for collection, evaluation and recording, and the technicians maintaining the robots. Appendix A provides contributions by activity.

Appendix A

The appendix documents contributions, attention layouts, world-model training, inference costs, the human comparison and task scoring criteria. These details delimit the headline generalization claims.

A-A Contributions

The Physical Intelligence team divides contributions across data collection and operations, annotation and supplemental data, policy research and training, policy infrastructure, hardware, robot infrastructure, and writing/illustration. The edition’s byline uses all 87 names in the reviewed v2 source; earlier imported name variants remain available as search aliases.

A-B Attention pattern

Figure 19 retains the full attention masks. Observation views share bidirectional attention; goal views add a separate block. Training FAST tokens and continuous flow actions do not attend to each other. Classifier-free guidance packs positive and negative branches into one sequence without mutual attention. The world model uses ViT observation tokens plus VAE encodings of current and noisy goal images, with a three-group guidance arrangement.

Original figure
Fig. 19: The π0.7\pi_{0.7} model and its world model (for generating subgoal images) use several different nontrivial attention patterns during training and inference. From top left: in absence of image goals we use the same attention patterns as in π0.5\pi_{0.5}, with global bidirectional attention between embeddings for all memory-aware image views. Note that the FAST tokens (only available at training time) and the flow actions do not attend to each other. When image goals are present, we include them as an additional block-causal bidirectional block, after the text prompt. When we do classifier-free guidance (at inference time), we pack both the positive and negative example into the same sequence for efficient inference by constructing an “attention tree” with two branches (positive and negative), which do not attend to one another. Following BAGEL, at training time the world model receives three copies of the image, each of which is block-bidirectional within the multiview group: current observation, encoded with ViT; current observation encoded with VAE; noisy image goal encoded with VAE. The world model uses a similar inference-time CFG trick, albeit with a more complex mask because it has three CFG groups rather than two.

A-C Training of the world model

The BAGEL-derived world model is trained on robot/human segments with high-quality temporal language labels, mixed with image-editing and video data. Its understanding and generation backbones each have 7B parameters. ViT inputs use 448×336 and VAE inputs/targets 512×384 because their patch sizes differ. Accurate temporal segmentation is important for goal quality; runtime regeneration uses a four-second interval.

Original figure
Fig. 20: Joint vs. end-effector control for prior models on cross-embodiment tasks. We compare joint-space and end-effector (EE) control for baseline policies across a range of tasks, observing no substantial difference in performance between the two control modes.
Original figure
Fig. 21: Operator experience in the human subject study. Box plots show teleoperation experience (in hours) of the ten recruited operators across three categories: UR5e (target robot), the static bimanual robot (source robot), and all robots combined. The selected operators rank within the top 2% of our operator fleet in terms of teleoperation experience.
Original figure
Fig. 22: Comparison of π0.7\pi_{0.7} (GC) and human. We find that π0.7\pi_{0.7} (GC) achieves competitive performance compared to the human operators, in the shirt folding task with the UR5e bimanual platform.

A-D Inference speed and optimization

The VLA and high-level policy use one H100; minimal VLA inference is reported at 38 ms, rising to 127 ms with memory and image goals. The separate 14B goal generator uses four H100s with tensor parallelism, eight-bit matrix multiplications and modified SageAttention, producing twenty-five-step guided generations in 1.25 seconds. Robot execution continues asynchronously during goal generation. These hardware and latency details qualify the description of the world model as lightweight.

A-E Comparison of action spaces in cross-embodiment transfer

Across the baseline cross-embodiment tasks, end-effector control provides no clear systematic advantage over joint-space control. The main study therefore keeps joint-space control for a consistent comparison. This does not establish equivalence for every robot or task.

A-F Human subject study for cross-embodiment shirt folding

Ten operators from the most experienced 2% of the fleet, averaging about 375 hours across robots, attempt UR5e shirt folding three times each with no warm-up before the first attempt. They share the policy’s flattened-shirt start, time limit and scoring rubric. The source reports thirty trials and human averages of 90.9% progress/80.6% success, versus 85.6%/80% for the goal-conditioned policy. The reported 80.6% is retained as published rather than reconstructed as a simple integer fraction of thirty; the aggregation details are not fully specified. Similar measured scores do not constitute an equivalence test.

A-G Detailed task descriptions and scoring rubric

The rubrics distinguish complete success from partial progress. Laundry requires a neat fold and stack; espresso requires the specified dispensing, tamping, extraction and serving sequence; box building requires an intact folded box. Sandwich preparation is scored out of 9, shirt inversion out of 7, doorway traversal out of 3, zucchini slicing out of 3, and peeling/cleanup out of 9. Trash replacement has twelve points split among accessing the bin, removing the old bag, installing the new one and returning the bin. Memory tasks score mug swapping out of 4, hidden-object retrieval and drawer closure out of 4, two coffee scoops and lid closure out of 5, and window cleaning/disposal out of 5.

Reverse bussing scores twelve correctly reassigned objects; the reversed appliance task gives six points for food transfer and door operations. Table setting scores seven items, with possible penalties for critically incorrect placements; backpack insertion scores 3, nested containers/lids 6, and bagging two shirts 4. Shirt folding gives three alignment-based fold points plus up to three quality points; all six are required for success. New-task rubrics score a fully depressed French-press plunger out of 1, rice scooping out of 3, loading an air fryer out of 4, unloading all food onto a plate out of 2, and bagel toasting/serving out of 7. These definitions explain why high normalized progress can coexist with lower complete-task success.

Concept demonstration

Open image ↗