Literature
Display
← All papers

EmbodiSteer: Steering Embodiment-Agnostic Visuomotor Policies with Joint-Space Guidance for Zero-Shot Cross-Embodiment Deployment

Shihefeng Wang · Kangchen Lv · Mingrui Yu · Xiang Li

Preprint posted · CoRL 2026 (accepted) ↗

Robotics11 min text · CondensedOriginal paper ↗
Contents

Overview

This edition covers the complete 29-page v2 dated 13 October 2026. It retains all method details, repeated evaluations, sensitivity studies, runtime measurements and failure analysis. The learned policy is unchanged at deployment; robot-specific geometry enters the sampling procedure.

Complete original figure with every panel, label and legend.
Figure 1: We present EmbodiSteer, an inference-time steering framework for embodiment-aware deployment of embodiment-agnostic visuomotor policies. Given a trained Cartesian policy (left), EmbodiSteer lifts the sampling process into the target robot’s joint space and incorporates robot embodiment and obstacle guidance during denoising (middle), enabling zero-shot whole-body collision-aware execution across diverse robots without finetuning (right).

At a glance

EmbodiSteer maintains a joint trajectory while querying a frozen Cartesian diffusion policy through forward kinematics. Damped Jacobian updates follow the policy’s denoising residual, then a local quadratic program moves the whole arm away from obstacles while penalizing end-effector deviation.

Across nine simulated arms, the main evaluation improves success from 35.7% to 64.2% and reduces collision occurrence from 57.6% to 11.5%. Across two physical arms and three tasks, success rises from 25/60 to 47/60 and collisions fall from 58/60 to 4/60. These are empirical results with modeled obstacles, not a formal safety guarantee.

1 Introduction

Cartesian end-effector actions make heterogeneous robot and UMI handheld-gripper demonstrations share one action representation, but do not describe the deployed arm’s links or feasible joint configurations. Reproducing a gripper path can therefore cause elbow or forearm collisions.

The paper separates this execution problem from learning a new embodiment-specific policy. It preserves a pretrained Diffusion Policy and introduces the target kinematics and whole-body constraints during sampling, requiring no target-arm demonstrations or finetuning.

2 Related Work

Cross-embodiment approaches align action representations through decoders, reusable skills or embodiment conditioning; XMoP instead learns a transferable whole-body motion planner. EmbodiSteer adds body awareness to an existing Cartesian manipulation policy at deployment.

Inference guidance can bias, filter or optimize diffusion samples. JM2D combines learned planners and model-based constraints through joint sampling, while EADP emphasizes controller tracking for aerial manipulation. This work targets arm-body obstacle avoidance while retaining the manipulation policy’s task motion.

Safety filters, control barrier functions, collision-guided diffusion and dedicated whole-body planning policies provide related mechanisms. EmbodiSteer uses a local CBF-inspired correction, without learning a new collision-aware trajectory representation or claiming a global certificate.

3 Preliminaries: Cartesian Diffusion Policy

The base model learns to predict Gaussian noise added to demonstration action chunks, then samples by reverse denoising. Each action contains relative translation, a continuous six-dimensional rotation representation and a gripper command; the main equations omit the gripper for brevity.

Poses are relative to the chunk-start end effector, so the action semantics are independent of arm joint count and ordering. Equation 1 presents a DDPM update; implementation uses a 16-step DDIM scheduler.

Equation 1
At−1=1αt​(At−1−αt1−α¯t​ϵθ​(At,o,t))+σt​z,z∼𝒩⁡(0,I).A_{t-1}=\frac{1}{\sqrt{\alpha_{t}}}\left(A_{t}-\frac{1-\alpha_{t}}{\sqrt{1-\bar{\alpha}_{t}}}{\epsilon}_{\theta}(A_{t},o,t)\right)+\sigma_{t}{z},\quad{z}\sim\mathcal{N}(0,I). (1)
Complete original figure with every panel, label and legend.
Figure 2: Overview of EmbodiSteer. (a) A trained Cartesian policy is lifted into the target-robot joint space for inference-time sampling. (b) CBF-inspired guidance steers sampled joint trajectories away from whole-body collisions while preserving end-effector behavior. (c) Visualization of real-world deployment across diverse robot embodiments on constrained manipulation tasks.
Equations
At=α¯t​A0+1−α¯t​ϵtA_{t}=\sqrt{\bar{\alpha}_{t}}A_{0}+\sqrt{1-\bar{\alpha}_{t}}\,\epsilon_{t}
ℒ=MSELoss⁡(ϵt,ϵθ​(At,o,t))\mathcal{L}=\mathrm{MSELoss}(\epsilon_{t},\epsilon_{\theta}(A_{t},o,t))

4 Method

Inference alternates two operations: use the frozen Cartesian denoiser to propose task progress, then correct the corresponding joint states using target-robot geometry. Algorithm 1 retains the complete procedure.

4.1 Problem Formulation

The target robot has a joint vector at each action-horizon step. A whole-body signed-distance function measures clearance from known obstacles, and the desired condition is clearance at least a positive margin at every sampled state.

This waypoint condition does not itself certify motion between states. The implemented distance is a smooth approximation over collision spheres, and both the nonlinear geometry and joint correction are subsequently approximated.

Equation 2
h(qi)=SDF(ℰ(qi),𝒪)≥dsafe,∀i=1,…,H,h(q_{i})=\mathrm{SDF}(\mathcal{E}(q_{i}),\mathcal{O})\geq d_{\mathrm{safe}},\qquad\forall\,i=1,\dots,H, (2)
Complete original pseudocode with exact line numbers, comments and indentation.
Algorithm 1 EmbodiSteer: Joint-Space Inference with Whole-Body Collision Guidance
Selectable original pseudocode
Input: Observation oo, obstacles 𝒪\mathcal{O}, embodiment ℰ\mathcal{E}
Output: Joint action chunk Q0Q_{0}
1 Sample Cartesian noise aT,ia_{T,i} and initialize QT=[qT,1,…,qT,H]Q_{T}=[q_{T,1},\dots,q_{T,H}] using  (3);
2 for t=T,…,1t=T,\dots,1 do
   // Joint Space Denoising
    3 Map QtQ_{t} to Cartesian action AtA_{t} using FKℰ\mathrm{FK}_{\mathcal{E}} ; // Kinematic mapping
    4 At−1←DiffusionStep​(ϵθ​(At,t,o),t,At)A_{t-1}\leftarrow\text{DiffusionStep}(\epsilon_{\theta}(A_{t},t,o),\,t,\,A_{t}); // Frozen Cartesian policy
    5 [ξt,i]←𝒯pose​(At−1​At−1)\left[\xi_{t,i}\right]\leftarrow\mathcal{T}_{\mathrm{pose}}\left(A_{t}^{-1}A_{t-1}\right); // Cartesian denoising residual
    6 Update joint space sample Qt−1diffQ_{t-1}^{\mathrm{diff}} using  (4); // Jacobian residual update
   // Whole-Body Collision-Aware Guidance
    7 Solve  (6) in batch to get solution Δ​Qt−1cbf\Delta Q_{t-1}^{\mathrm{cbf}}; // Solve QP
    8 Apply the solution Δ​Qt−1cbf\Delta Q_{t-1}^{\mathrm{cbf}} to Qt−1diffQ_{t-1}^{\mathrm{diff}} using  (7) to get Qt−1Q_{t-1}; // Apply the guidance
9 end for
10 return Q0Q_{0}

4.2 Joint-Space Denoising with Pretrained Cartesian Diffusion Policy

Repeated inverse-kinematics solves within diffusion are expensive. Instead, map each local six-dimensional pose residual through a damped Jacobian pseudoinverse. Damping limits amplification near singularities; clipping bounds each per-joint update.

Initialize around the current configuration by projecting Cartesian Gaussian noise into joint perturbations, then scaling and clipping. This is a local kinematic construction, rather than an exact preservation proof for the original Cartesian sampling distribution.

Equations 3
qT,i=qstart+α⋅clip⁡(Jℰ+​(qstart)​ξT,i,−Δ​qmax,Δ​qmax),ξT,i=𝒯pose​(aT,i).q_{T,i}=q^{\mathrm{start}}+\alpha\cdot\mathrm{clip}\bigl(J_{\mathcal{E}}^{+}(q^{\mathrm{start}})\,\xi_{T,i},\,-\Delta q_{\mathrm{max}},\,\Delta q_{\mathrm{max}}\bigr),\quad\xi_{T,i}=\mathcal{T}_{\mathrm{pose}}(a_{T,i}). (3)
Jℰ+​(q)=Jℰ​(q)⊤​(Jℰ​(q)​Jℰ​(q)⊤+λpinv​I)−1J_{\mathcal{E}}^{+}(q)=J_{\mathcal{E}}(q)^{\top}\bigl(J_{\mathcal{E}}(q)J_{\mathcal{E}}(q)^{\top}+\lambda_{\mathrm{pinv}}I\bigr)^{-1}

Joint-Space Denoising Update.

Forward kinematics converts the current noisy joint chunk back into the denoiser’s relative Cartesian representation. The frozen network proposes the next Cartesian chunk. Its relative pose residual becomes a translation/rotation twist, which the damped Jacobian maps into a clipped joint increment.

Equations 4
qt−1,idiff=qt,i+clip⁡(Jℰ+​(qt,i)​ξt,i,−Δ​qmax,Δ​qmax).q_{t-1,i}^{\mathrm{diff}}=q_{t,i}+\mathrm{clip}\bigl(J_{\mathcal{E}}^{+}(q_{t,i})\xi_{t,i},\,-\Delta q_{\mathrm{max}},\,\Delta q_{\mathrm{max}}\bigr). (4)
At−1=DiffusionStep⁡(ϵθ​(At,t,o),t,At)A_{t-1}=\mathrm{DiffusionStep}\bigl(\epsilon_{\theta}(A_{t},t,o),t,A_{t}\bigr)
ξt,i=𝒯pose​(at,i−1​at−1,i)\xi_{t,i}=\mathcal{T}_{\mathrm{pose}}(a_{t,i}^{-1}a_{t-1,i})

4.3 CBF-Inspired Whole-Body Collision-Aware Guidance

Plain collision-cost gradients can move an elbow away from an obstacle while displacing the gripper from a precise grasp or contact. The guidance objective therefore explicitly penalizes task-space motion, rather than treating all joint-space escape directions equally.

Equation
qt−1,i=qt−1,idiff−ρt​∇qℒcoll​(qt−1,idiff)q_{t-1,i}=q_{t-1,i}^{\mathrm{diff}}-\rho_{t}\nabla_{q}\mathcal{L}_{\mathrm{coll}}\left(q_{t-1,i}^{\mathrm{diff}}\right)

CBF-Inspired Guidance Formulation.

For each horizon state independently, the nonlinear objective balances weighted end-effector deviation and joint deviation subject to a clearance margin. Position and rotation receive separate weights; joint regularization discourages unnecessarily large corrections.

Equation 5
minqt−1,i⁡12​‖FK​(qt−1,idiff)−1​FK​(qt−1,i)‖W2+12​λcbf​‖qt−1,i−qt−1,idiff‖2,s.t.h⁡(qt−1,i)≥dsafe,\min_{q_{t-1,i}}\frac{1}{2}\left\|\mathrm{FK}\left(q_{t-1,i}^{\mathrm{diff}}\right)^{-1}\mathrm{FK}\left(q_{t-1,i}\right)\right\|_{W}^{2}+\frac{1}{2}\lambda_{\mathrm{cbf}}\left\|q_{t-1,i}-q_{t-1,i}^{\mathrm{diff}}\right\|^{2},\quad\mathrm{s.t.}\,\,h(q_{t-1,i})\geq d_{\text{safe}}, (5)

QP Linearization for Efficient Update.

Linearizing forward kinematics and clearance yields a positive regularized quadratic cost with one linear inequality. Its matrix combines the Jacobian task metric and an identity regularizer; a scheduled strength determines how aggressively the margin deficit is corrected.

The single-constraint problem admits a closed-form solution. The resulting joint increment is clipped again before use. Linearization, smoothing, the numerical stabilizer and clipping mean this is practical collision guidance, not certified feasibility of the original nonlinear constraint.

Equations 6–7
minΔ​qi⁡12​Δ​qt−1,i⊤​Hℰ​(qt−1,idiff)​Δ​qt−1,i,s.t.∇qh​(qt−1,idiff)⊤​Δ​qt−1,i≥γt​[dsafe−h⁡(qt−1,idiff)],\min_{\Delta q_{i}}\,\frac{1}{2}\Delta q_{t-1,i}^{\top}H_{\mathcal{E}}\left(q_{t-1,i}^{\mathrm{diff}}\right)\Delta q_{t-1,i},\,\quad\mathrm{s.t.}\,\,\nabla_{q}h\left(q_{t-1,i}^{\mathrm{diff}}\right)^{\top}\Delta q_{t-1,i}\geq\gamma_{t}\left[d_{\mathrm{safe}}-h\left(q_{t-1,i}^{\mathrm{diff}}\right)\right], (6)
qt−1,i=qt−1,idiff+clip⁡(Δ​qt−1,icbf,−Δ​qmaxcbf,Δ​qmaxcbf).q_{t-1,i}=q_{t-1,i}^{\mathrm{diff}}+\mathrm{clip}(\Delta q_{t-1,i}^{\mathrm{cbf}},-\Delta q_{\max}^{\mathrm{cbf}},\Delta q_{\max}^{\mathrm{cbf}}). (7)
Complete original figure with every panel, label and legend.
Figure 3: We evaluate EmbodiSteer on three manipulation tasks requiring both task completion and whole-body obstacle avoidance. Cartesian policies are trained from obstacle-free floating-gripper demonstrations and deployed zero-shot on 9 robot embodiments with test-time obstacles.
Equation
Hℰ​(qt−1,idiff)=Jℰ​(qt−1,idiff)⊤​W​Jℰ​(qt−1,idiff)+λcbf​IH_{\mathcal{E}}\left(q_{t-1,i}^{\mathrm{diff}}\right)=J_{\mathcal{E}}\left(q_{t-1,i}^{\mathrm{diff}}\right)^{\top}WJ_{\mathcal{E}}\left(q_{t-1,i}^{\mathrm{diff}}\right)+\lambda_{\mathrm{cbf}}I

5 Results

The evaluation separates transfer without obstacles, obstacle-aware execution in simulation, and physical deployment. Success, reward and collision occurrence measure different outcomes and should be read together.

5.1 Simulation Results

Each of three ManiSkill tasks uses 200 obstacle-free floating-gripper demonstrations. The same wrist camera and gripper mount are retained across nine arms; the learned policy receives wrist RGB and end-effector state. Obstacles appear only at deployment in the main benchmark.

Without obstacles, Cartesian and unguided joint sampling average 90.0% and 90.7% success. With obstacles, EmbodiSteer reaches 64.2% success and 11.5% collisions versus 57.0% and 17.9% for a one-shot post-hoc CBF correction. Candidate sampling offers only a modest gain, while unregularized collision gradients can substantially hurt precise tasks.

Three evaluation seeds using the same trained policies give 63.3±1.4% success and 11.7±.8% collisions for EmbodiSteer, versus 36.0±.2% and 56.8±.9% for Cartesian execution. These deviations quantify evaluation-seed variation, not independent policy training runs.

A JM2D-style adaptation takes 8.65 seconds per chunk with 16 proposals and 16 denoising steps, versus .254 seconds for EmbodiSteer on the matched RTX 4090 setup. Only computational feasibility is compared; no full task-performance result is reported for that adaptation. Increasing demonstration quantity or waypoint diversity improves performance under guidance-induced perturbations.

Table 1: Quantitative results on manipulation tasks with obstacles, averaged over 9 robot embodiments. TSR is task success rate (%), RWD is maximum reward per episode, and COR is collision occurrence rate (%). Best results among obstacle-present methods are shown in bold. Repeated-seed aggregate results are reported in Tab. 2.
Obstacles Method PlaceToast TurnOnFaucet MakeCoffee Average
TSR↑\uparrow RWD↑\uparrow COR↓\downarrow TSR↑\uparrow RWD↑\uparrow COR↓\downarrow TSR↑\uparrow RWD↑\uparrow COR↓\downarrow TSR↑\uparrow RWD↑\uparrow COR↓\downarrow
w/o Obs. EE 96.8 .980 – 83.6 .937 – 89.7 .932 – 90.0 .950 –
Joint 94.4 .965 – 87.9 .957 – 89.7 .929 – 90.7 .950 –
w/ Obs. EE 43.4 .634 52.6 41.6 .752 61.8 22.2 .456 58.3 35.7 .614 57.6
EE w/ Sampling 47.1 .661 51.3 43.9 .767 53.2 24.1 .478 60.0 38.4 .635 54.8
EE w/ CBF 65.8 .764 15.3 56.3 .797 29.9 48.9 .606 8.6 57.0 .722 17.9
Joint w/ CG 57.7 .687 0.2 10.3 .403 59.6 22.4 .424 40.6 30.1 .505 33.5
EmbodiSteer 74.8 .829 4.6 60.6 .833 27.1 57.2 .670 2.8 64.2 .777 11.5
Complete original figure with every panel, label and legend.
Figure 4: Per-embodiment comparison on the PlaceToast task, where the task success rate (left) and collision rate (right) across five representative robot embodiments are reported.
Complete original figure with every panel, label and legend.
Figure 5: Ablation for the constraint strength on TurnOnFaucet.
Table 2: Simulation results with obstacles, averaged over three tasks and nine robot embodiments. Entries report aggregate mean ±\pm standard deviation across three evaluation seeds.
Method TSR (%) ↑\uparrow RWD ↑\uparrow COR (%) ↓\downarrow
EE 36.0±0.236.0\pm 0.2 .614±.004.614\pm.004 56.8±0.956.8\pm 0.9
EE w/ Sampling 37.7±0.537.7\pm 0.5 .621±.012.621\pm.012 53.1±1.353.1\pm 1.3
EE w/ CBF 57.2±0.657.2\pm 0.6 .726±.005.726\pm.005 18.3±0.318.3\pm 0.3
Joint w/ CG 30.3±0.530.3\pm 0.5 .503±.003.503\pm.003 33.5±0.633.5\pm 0.6
EmbodiSteer 63.3±1.4\mathbf{63.3\pm 1.4} .771±.011\mathbf{.771\pm.011} 11.7±0.8\mathbf{11.7\pm 0.8}
Complete original figure with every panel, label and legend.
Figure 6: Demonstration coverage on MakeCoffee: varying the number of demonstrations (left) and motion-planning waypoint randomization relative to the original data-generation setting (right).

5.2 Real-World Experiments

UR5 and Franka Panda share one UMI-trained Cartesian checkpoint per task, with 200 obstacle-free demonstrations and no robot-specific finetuning. Each robot-task pair has ten no-obstacle trials and ten obstacle trials.

The base policy succeeds 49/60 times without obstacles. Under obstacles, success is 25/60 with 58/60 collisions; guidance yields 47/60 successes and 4/60 collisions. The changes are +36.7 and −90.0 percentage points. Movable obstacles permit task completion after contact, so success and collision are annotated independently.

A separate runtime breakdown measures 103 ms for one guided call on an RTX 4070 Ti SUPER, approximately 9.7 calls per second. This measurement differs from the synchronized RTX 4090 comparison and should not be directly combined with it.

Complete original figure with every panel, label and legend.
Figure 7: Real-world deployment of an arm-agnostic Cartesian policy on UR5 and Panda. Without guidance, the base policy often preserves task intent but collides with obstacles (yellow region), whereas EmbodiSteer steers joint trajectories to avoid arm-body collisions and complete tasks.

6 Limitations & Conclusion

Guidance is local: it becomes active near the modeled clearance margin and cannot choose globally collision-free routes far in advance. Nonlinear geometry, clipped updates and strong corrections can leave collisions unresolved or move observations outside the policy’s training distribution.

The contribution is a training-free bridge between embodiment-agnostic action learning and embodiment-specific execution. Broader demonstrations help, but known geometry and local linearization remain material restrictions.

Acknowledgments

The authors acknowledge discussions, hardware and experimental assistance, and reviewer feedback.

Appendix A Method Details

The appendix makes the action representation, signed-distance convention and one-constraint quadratic-program solution explicit. All original numbered expressions and substantive inline definitions are retained.

A.1 Cartesian Action Conversion and Pose-Twist Mapping

The chunk-start pose converts relative translation and six-dimensional rotation into world-frame end-effector poses. The shorthand inverse action product denotes the relative rigid transform, not inversion of a raw nine-dimensional vector.

The pose-twist operator concatenates relative translation and the rotation-vector logarithm. Initialization uses the current chunk-start joints; damped Jacobian updates and per-joint clipping control the error of this local approximation.

Equations 8–10
Φ⁡(ai,T0)=(pi,Ri),pi=p0+R0​δ​pi,Ri=R0​ϕ6​D​(ri),\Phi(a_{i};T_{0})=(p_{i},R_{i}),\qquad p_{i}=p_{0}+R_{0}\delta p_{i},\qquad R_{i}=R_{0}\phi_{\mathrm{6D}}(r_{i}), (8)
Φ​(at,i,T0)−1​Φ​(at−1,i,T0)=(Δ​pt,i,Δ​Rt,i).\Phi(a_{t,i};T_{0})^{-1}\Phi(a_{t-1,i};T_{0})=(\Delta p_{t,i},\Delta R_{t,i}). (9)
ξt,i=𝒯pose​(Φ​(at,i,T0)−1​Φ​(at−1,i,T0))=[Δ​pt,iLog⁡(Δ​Rt,i)],\xi_{t,i}=\mathcal{T}_{\mathrm{pose}}\left(\Phi(a_{t,i};T_{0})^{-1}\Phi(a_{t-1,i};T_{0})\right)=\begin{bmatrix}\Delta p_{t,i}\\ \mathrm{Log}(\Delta R_{t,i})\end{bmatrix}, (10)

A.2 Whole-Body SDF Representation

cuRobo approximates links with collision spheres and queries known cuboid obstacles. Its signed distance is positive for penetration, opposite the main paper’s positive-clearance convention. Exact worst clearance is the negative maximum sphere distance.

Guidance instead uses the negative normalized log-sum-exp of the four largest sphere distances, with temperature 20. This smooth maximum is at most the true maximum, so the resulting clearance can be optimistic; the source describes the bias without bounding its safety effect. Unknown-obstacle perception is outside scope.

Equations 11–13
dmax​(q)=maxj⁡dj​(q).d_{\max}(q)=\max_{j}d_{j}(q). (11)
d~max​(q)=m⁡(q)+1τ​log⁡(1k​∑j∈𝒦⁡(q)exp⁡(τ⁡[dj​(q)−m⁡(q)])),\widetilde{d}_{\max}(q)=m(q)+\frac{1}{\tau}\log\left(\frac{1}{k}\sum_{j\in\mathcal{K}(q)}\exp\left(\tau\left[d_{j}(q)-m(q)\right]\right)\right), (12)
h⁡(q)=−d~max​(q).h(q)=-\widetilde{d}_{\max}(q). (13)
Complete original figure with every panel, label and legend.
Figure 8: Whole-body SDF representation. The left image shows the real scene. The right image shows the corresponding cuRobo scene used for SDF queries, where the robot is represented by collision spheres and the obstacle is represented as a cuboid.

A.3 CBF-Inspired QP Details

The regularized Jacobian metric gives a one-constraint QP. The schedule follows constraint-aware diffusion guidance: a sigmoid with slope 50 and transition .7 is weak at early noisy steps and approaches the base strength later. Already-safe states receive zero correction through a nonnegative constraint deficit.

The KKT derivation yields a correction along the inverse-metric clearance gradient, scaled by the deficit divided by its quadratic norm. A small denominator stabilizer and subsequent clipping modify the ideal solution. Only arm joints are corrected; gripper commands remain unchanged.

Equations 14–16
minΔ​q⁡12​Δ​q⊤​H​Δ​q,s.t.a⊤​Δ​q≥b,\min_{\Delta q}\frac{1}{2}\Delta q^{\top}H\Delta q,\qquad\mathrm{s.t.}\quad a^{\top}\Delta q\geq b, (14)
a=∇qh​(qt−1,idiff),b=γt​[dsafe−h⁡(qt−1,idiff)],H=Jℰ⊤​W​Jℰ+λcbf​I.a=\nabla_{q}h(q_{t-1,i}^{\mathrm{diff}}),\qquad b=\gamma_{t}\left[d_{\mathrm{safe}}-h(q_{t-1,i}^{\mathrm{diff}})\right],\qquad H=J_{\mathcal{E}}^{\top}WJ_{\mathcal{E}}+\lambda_{\mathrm{cbf}}I. (15)
γt=γ⋅σ⁡(β⋅(c−tT−1)),σ⁡(z)=11+exp⁡(−z),\gamma_{t}=\gamma\cdot\sigma\!\left(\beta\cdot\left(c-\frac{t}{T-1}\right)\right),\qquad\sigma(z)=\frac{1}{1+\exp(-z)}, (16)
Complete original figure with every panel, label and legend.
Figure 9: Scheduled constraint strength γt\gamma_{t} over reverse denoising steps. The guidance is weak in early noisy steps and approaches the base scale γ\gamma in later steps.
Equations 17–18
ℒ⁡(Δ​q,μ)=12​Δ​q⊤​H​Δ​q+μ⁡(b¯−a⊤​Δ​q),μ≥0.\mathcal{L}(\Delta q,\mu)=\frac{1}{2}\Delta q^{\top}H\Delta q+\mu(\bar{b}-a^{\top}\Delta q),\qquad\mu\geq 0. (17)
Δ​q⋆={0,b≤0,ba⊤​H−1​a+ε​H−1​a,b>0,\Delta q^{\star}=\begin{cases}0,&b\leq 0,\\[6.00006pt] \dfrac{b}{a^{\top}H^{-1}a+\varepsilon}\,H^{-1}a,&b>0,\end{cases} (18)

Appendix B Implementation Details

Implementation details specify the shared visual policy, deployment parameters, runtime accounting and all baseline procedures.

B.1 Diffusion Policy Architecture

A pretrained CLIP ViT-B/16 visual encoder is finetuned jointly with a conditional one-dimensional U-Net. Two observations condition sixteen ten-dimensional actions: three translation, six rotation and one gripper coordinate. Deployment freezes the model and uses sixteen DDIM denoising steps.

B.2 Hyperparameters

Table 3 retains all training and deployment settings. Policies train for 120 epochs with AdamW and EMA; control runs at 10 Hz. Sixteen actions are executed before replanning in simulation, six on hardware.

Default Jacobian damping is .001, projected-noise scale .1, denoising joint clip .5, QP regularization .01 and guidance clip .1. Task-dependent safety margins are .05–.10, with position/rotation weights 1/.1 and four critical collision spheres.

Table 3: Training and deployment hyperparameters.
Quantity Value Notes
Observation horizon 2 Diffusion policy input setting.
Action horizon HH 16 Predicted action chunk length.
Execution horizon 16 in simulation, 6 in real world. Number of actions executed before replanning.
Visual backbone CLIP ViT-B/16 vit_base_patch16_clip_224.openai loaded through timm, pretrained and finetuned.
U-Net channels [256, 512, 1024] Conditional 1D U-Net downsampling dimensions.
Diffusion step embedding 128 Timestep embedding dimension.
Diffusion training timesteps 50 Squared-cosine beta schedule and epsilon prediction.
Inference denoising steps 16 DDIM reverse diffusion steps at deployment.
Control frequency 10 Simulation and real-world control rate.
Training epochs 120 Used for both simulation and real-world policies.
Batch size 128 in simulation, 64 in real world. Training batch size for simulation and real-world checkpoints.
Optimizer AdamW Learning rate 3×10−43\times 10^{-4}, betas (0.95,0.999)(0.95,0.999), weight decay 10−610^{-6}.
Learning-rate schedule Cosine 2000 warmup steps.
EMA 0.9999 max decay EMA enabled during training.
Image augmentation Crop + color jitter Random crop ratio 0.95; brightness 0.3, contrast 0.4, saturation 0.5, hue 0.08.
Noise initialization scale α\alpha 0.1 Scale for Jacobian-projected noise.
Jacobian damping λpinv\lambda_{\mathrm{pinv}} 0.001 Used in Jℰ+J_{\mathcal{E}}^{+}.
Joint update clip Δ​qmax\Delta q_{\max} 0.5 Used in joint denoising update.
Safety margin dsafed_{\mathrm{safe}} 0.05∼\sim0.10 Task-dependent; set to 0.05, 0.07, or 0.10 by obstacle layout.
SDF aggregation top-kk 4 Number of critical collision spheres.
Smoothmax temperature τ\tau 20.0 Temperature in  (12).
Base constraint strength γ\gamma 1.0 Base multiplier in  (16).
QP regularization λcbf\lambda_{\mathrm{cbf}} 0.01 Joint-space regularization.
Task position weight 1.0 Position rows in Jℰ⊤​W​JℰJ_{\mathcal{E}}^{\top}WJ_{\mathcal{E}}.
Task rotation weight 0.1 Rotation rows in Jℰ⊤​W​JℰJ_{\mathcal{E}}^{\top}WJ_{\mathcal{E}}.
CBF update clip Δ​qmaxcbf\Delta q_{\max}^{\mathrm{cbf}} 0.1 Clipping applied to the guidance update.

B.3 Runtime Breakdown

One RTX 4070 Ti SUPER guided call totals 103 ms: the base encoder/denoiser consumes 46.8 ms and added lifting/guidance/formatting/overhead 56.2 ms. The full original breakdown and pie chart remain visible. This is inference latency for a call containing all sixteen reverse steps, not execution time for the whole action chunk.

Complete original figure with every panel, label and legend.
Figure 10: Runtime breakdown for one guided joint-space inference call on an RTX 4070 Ti SUPER.
Selectable original table
Component Included operations ms %
Base Cartesian policy components
Diffusion U-Net U-Net denoising, 16 steps 41.2 39.9
Observation Encoder Image/state observation encoding 5.6 5.4
Base policy subtotal Measured base-policy components 46.8 45.4
Additional EmbodiSteer components
Kinematic Mapping FK, Jacobian, DLS, twist 13.5 13.1
CBF Guidance CBF autograd, QP, linearization setup 11.5 11.2
Collision SDF Query Sphere FK, SDF query, world setup 2.8 2.7
Pose / Norm. / Scheduler Pose conversion, normalizer, scheduler step 11.1 10.8
I/O and Action Formatting Obs. preprocessing, joint tensor, action conversion 1.0 1.0
Other Runtime Overhead Uninstrumented loop/PyTorch overhead 16.3 15.8
Added subtotal Joint-space lifting and guidance overhead 56.2 54.6
Full total Guided joint-space inference 103.0 100.0

B.4 Baseline Implementation Details

All baselines share the task checkpoint. EE samples Cartesian actions; Joint changes the denoising variable without guidance. EE with Sampling selects the largest-clearance realization from sixteen candidates; EE with CBF performs IK and one post-hoc QP correction. Joint with CG adds collision gradients without the task-preserving quadratic metric.

Table 4: Baseline implementation details.
Method Implementation
EE Runs the Cartesian diffusion policy in its original action space and executes the predicted end-effector actions without collision-aware guidance.
Joint Uses the same joint-space denoising procedure as EmbodiSteer, including FK queries to the frozen Cartesian denoiser and damped Jacobian updates, but does not apply collision guidance.
EE w/ Sampling Keeps Cartesian denoising unchanged, samples multiple candidate action chunks, realizes them through the target robot, and selects the candidate with the largest clearance. Number of candidates: 16.
EE w/ CBF Keeps Cartesian denoising unchanged, solves IK for the generated action chunk, and applies a one-shot CBF-QP correction after generation.
Joint w/ CG Adds collision-cost gradient guidance during joint-space denoising. Unlike EmbodiSteer, the update is not explicitly regularized by the task-space QP objective, so whole-body collision gradients can perturb the end-effector trajectory.

Appendix C Simulation Results

The simulation appendix preserves all per-robot measurements, repeated evaluations, ablations, data-coverage studies, candidate-budget costs and trajectory diagnostics.

C.1 Task Definitions and Evaluation Protocol

PlaceToast, TurnOnFaucet and MakeCoffee use motion-planned floating-gripper training data and the same test obstacle layouts across nine arms. TSR is success-once rate, RWD is maximum episode reward, and COR records any collision; the table reports COR only when obstacles are present.

C.2 Full Quantitative Results

Table 5 retains every arm/task/baseline entry. Improvements are not uniform: for example, guided TurnOnFaucet on Rizon4 still has 88% collision occurrence and only 15% success. Aggregate performance therefore does not establish reliable safety on each embodiment.

Table 5: Full per-robot simulation results. Obs. indicates whether obstacles are present during evaluation; TSR is success-once rate, RWD is maximum reward per episode, and COR is collision occurrence rate. COR is reported only for obstacle-present settings.
Obs. Method Robot TSR↑\uparrow RWD↑\uparrow COR↓\downarrow Robot TSR↑\uparrow RWD↑\uparrow COR↓\downarrow Robot TSR↑\uparrow RWD↑\uparrow COR↓\downarrow
PlaceToast
w/o
Obs.
UR5 100.0% 1.000 – Panda 100.0% 1.000 – xArm6 100.0% 1.000 –
xArm7 98.0% 0.989 – iiwa7 98.0% 0.989 – Gen3-6D 81.0% 0.879 –
Gen3-7D 98.0% 0.989 – Rizon4 99.0% 0.991 – Sawyer 97.0% 0.984 –
Avg. 96.8% 0.980 –
Joint
UR5 100.0% 1.000 – Panda 100.0% 1.000 – xArm6 98.0% 0.988 –
xArm7 99.0% 0.995 – iiwa7 98.0% 0.989 – Gen3-6D 73.0% 0.827 –
Gen3-7D 85.0% 0.907 – Rizon4 98.0% 0.983 – Sawyer 99.0% 0.995 –
Avg. 94.4% 0.965 –
w/
Obs.
UR5 39.0% 0.560 50.0% Panda 46.0% 0.585 54.0% xArm6 45.0% 0.676 52.0%
xArm7 48.0% 0.695 56.0% iiwa7 47.0% 0.689 49.0% Gen3-6D 36.0% 0.611 51.0%
Gen3-7D 43.0% 0.665 51.0% Rizon4 42.0% 0.552 51.0% Sawyer 45.0% 0.676 59.0%
Avg. 43.4% 0.634 52.6%
EE w/
Samp.
UR5 48.0% 0.626 62.0% Panda 54.0% 0.651 46.0% xArm6 55.0% 0.736 49.0%
xArm7 52.0% 0.717 50.0% iiwa7 44.0% 0.663 40.0% Gen3-6D 36.0% 0.613 47.0%
Gen3-7D 45.0% 0.677 53.0% Rizon4 48.0% 0.601 62.0% Sawyer 42.0% 0.661 53.0%
Avg. 47.1% 0.661 51.3%
EE w/
UR5 76.0% 0.797 11.0% Panda 64.0% 0.692 17.0% xArm6 75.0% 0.849 8.0%
xArm7 66.0% 0.796 14.0% iiwa7 62.0% 0.768 14.0% Gen3-6D 62.0% 0.766 5.0%
Gen3-7D 68.0% 0.809 13.0% Rizon4 50.0% 0.580 37.0% Sawyer 69.0% 0.816 19.0%
Avg. 65.8% 0.764 15.3%
Joint
w/ CG
UR5 35.0% 0.386 0.0% Panda 76.0% 0.789 0.0% xArm6 62.0% 0.752 0.0%
xArm7 67.0% 0.786 0.0% iiwa7 44.0% 0.628 1.0% Gen3-6D 54.0% 0.699 0.0%
Gen3-7D 57.0% 0.721 0.0% Rizon4 62.0% 0.665 0.0% Sawyer 62.0% 0.753 1.0%
Avg. 57.7% 0.687 0.2%
Ours
UR5 78.0% 0.807 0.0% Panda 82.0% 0.842 3.0% xArm6 77.0% 0.859 0.0%
xArm7 68.0% 0.805 0.0% iiwa7 84.0% 0.905 9.0% Gen3-6D 54.0% 0.712 0.0%
Gen3-7D 76.0% 0.853 1.0% Rizon4 82.0% 0.849 16.0% Sawyer 72.0% 0.831 12.0%
Avg. 74.8% 0.829 4.6%
TurnOnFaucet
w/o
Obs.
Panda 81.0% 0.943 – xArm6 99.0% 0.998 – xArm7 78.0% 0.937 –
UR5 90.0% 0.918 – iiwa7 87.0% 0.964 – Gen3-6D 96.0% 0.988 –
Gen3-7D 53.0% 0.880 – Rizon4 73.0% 0.822 – Sawyer 95.0% 0.987 –
Avg. 83.6% 0.937 –
Joint
UR5 88.0% 0.912 – Panda 90.0% 0.974 – xArm6 100.0% 1.000 –
xArm7 90.0% 0.971 – iiwa7 91.0% 0.975 – Gen3-6D 95.0% 0.988 –
Gen3-7D 63.0% 0.909 – Rizon4 87.0% 0.952 – Sawyer 87.0% 0.936 –
Avg. 87.9% 0.957 –
w/
Obs.
UR5 11.0% 0.417 86.0% Panda 23.0% 0.471 94.0% xArm6 26.0% 0.791 89.0%
xArm7 41.0% 0.823 61.0% iiwa7 66.0% 0.915 28.0% Gen3-6D 73.0% 0.931 41.0%
Gen3-7D 42.0% 0.860 40.0% Rizon4 12.0% 0.657 88.0% Sawyer 80.0% 0.904 29.0%
Avg. 41.6% 0.752 61.8%
EE w/
Samp.
UR5 6.0% 0.453 86.0% Panda 35.0% 0.554 87.0% xArm6 31.0% 0.811 84.0%
xArm7 32.0% 0.789 47.0% iiwa7 69.0% 0.911 18.0% Gen3-6D 74.0% 0.935 28.0%
Gen3-7D 50.0% 0.874 21.0% Rizon4 13.0% 0.665 90.0% Sawyer 85.0% 0.911 18.0%
Avg. 43.9% 0.767 53.2%
EE w/
UR5 33.0% 0.506 29.0% Panda 48.0% 0.661 46.0% xArm6 60.0% 0.857 36.0%
xArm7 55.0% 0.833 26.0% iiwa7 78.0% 0.945 9.0% Gen3-6D 87.0% 0.942 12.0%
Gen3-7D 48.0% 0.851 1.0% Rizon4 8.0% 0.632 97.0% Sawyer 90.0% 0.949 13.0%
Avg. 56.3% 0.797 29.9%
Joint
w/ CG
UR5 8.0% 0.448 88.0% Panda 7.0% 0.339 82.0% xArm6 6.0% 0.432 75.0%
xArm7 0.0% 0.239 65.0% iiwa7 12.0% 0.501 42.0% Gen3-6D 33.0% 0.667 30.0%
Gen3-7D 11.0% 0.481 23.0% Rizon4 2.0% 0.173 70.0% Sawyer 14.0% 0.350 61.0%
Avg. 10.3% 0.403 59.6%
Ours
UR5 32.0% 0.502 23.0% Panda 57.0% 0.802 42.0% xArm6 75.0% 0.900 30.0%
xArm7 52.0% 0.842 28.0% iiwa7 76.0% 0.926 17.0% Gen3-6D 90.0% 0.971 7.0%
Gen3-7D 61.0% 0.897 2.0% Rizon4 15.0% 0.744 88.0% Sawyer 87.0% 0.916 7.0%
Avg. 60.6% 0.833 27.1%
MakeCoffee
w/o
Obs.
UR5 94.0% 0.952 – Panda 94.0% 0.951 – xArm6 87.0% 0.917 –
xArm7 90.0% 0.943 – iiwa7 85.0% 0.913 – Gen3-6D 93.0% 0.960 –
Gen3-7D 94.0% 0.965 – Rizon4 88.0% 0.904 – Sawyer 82.0% 0.881 –
Avg. 89.7% 0.932 –
Joint
UR5 94.0% 0.951 – Panda 93.0% 0.943 – xArm6 91.0% 0.937 –
xArm7 90.0% 0.936 – iiwa7 92.0% 0.959 – Gen3-6D 95.0% 0.975 –
Gen3-7D 88.0% 0.927 – Rizon4 80.0% 0.833 – Sawyer 84.0% 0.900 –
Avg. 89.7% 0.929 –
w/
Obs.
UR5 8.0% 0.277 85.0% Panda 24.0% 0.408 60.0% xArm6 27.0% 0.536 70.0%
xArm7 28.0% 0.540 69.0% iiwa7 26.0% 0.548 59.0% Gen3-6D 25.0% 0.497 45.0%
Gen3-7D 24.0% 0.513 47.0% Rizon4 24.0% 0.385 44.0% Sawyer 14.0% 0.401 46.0%
Avg. 22.2% 0.456 58.3%
EE w/
Samp.
UR5 8.0% 0.243 87.0% Panda 24.0% 0.393 63.0% xArm6 24.0% 0.539 71.0%
xArm7 30.0% 0.577 64.0% iiwa7 39.0% 0.635 56.0% Gen3-6D 20.0% 0.490 56.0%
Gen3-7D 24.0% 0.523 52.0% Rizon4 23.0% 0.384 56.0% Sawyer 25.0% 0.516 35.0%
Avg. 24.1% 0.478 60.0%
EE w/
UR5 19.0% 0.314 8.0% Panda 62.0% 0.683 11.0% xArm6 58.0% 0.699 14.0%
xArm7 48.0% 0.615 2.0% iiwa7 66.0% 0.762 4.0% Gen3-6D 67.0% 0.769 6.0%
Gen3-7D 43.0% 0.574 4.0% Rizon4 39.0% 0.494 21.0% Sawyer 38.0% 0.543 7.0%
Avg. 48.9% 0.606 8.6%
Joint
w/ CG
UR5 13.0% 0.287 79.0% Panda 18.0% 0.333 51.0% xArm6 29.0% 0.528 56.0%
xArm7 31.0% 0.519 36.0% iiwa7 17.0% 0.385 28.0% Gen3-6D 31.0% 0.541 39.0%
Gen3-7D 33.0% 0.544 36.0% Rizon4 5.0% 0.183 13.0% Sawyer 25.0% 0.493 27.0%
Avg. 22.4% 0.424 40.6%
Ours
UR5 28.0% 0.387 3.0% Panda 75.0% 0.798 4.0% xArm6 68.0% 0.757 3.0%
xArm7 64.0% 0.756 2.0% iiwa7 73.0% 0.815 0.0% Gen3-6D 69.0% 0.795 2.0%
Gen3-7D 56.0% 0.662 1.0% Rizon4 38.0% 0.481 9.0% Sawyer 44.0% 0.574 1.0%
Avg. 57.2% 0.670 2.8%

C.3 Repeated Evaluations

Three evaluation seeds reuse each trained task policy. Table 6 gives task-level results before the aggregate mean and standard deviation in Table 2. This protocol tests evaluation randomness rather than retraining variability.

Table 6: Repeated EmbodiSteer evaluations with three different random seeds for each task.
Task Evaluation TSR (%) ↑\uparrow RWD ↑\uparrow COR (%) ↓\downarrow
MakeCoffee Run 1 57.2 .670 2.8
Run 2 57.9 .677 3.4
Run 3 53.8 .649 4.2
TurnOnFaucet Run 1 60.6 .833 27.1
Run 2 62.0 .843 23.9
Run 3 57.8 .809 29.3
PlaceToast Run 1 74.8 .829 4.6
Run 2 73.1 .818 5.4
Run 3 72.6 .810 4.7

C.4 Additional Ablations and Sensitivity Analyses

Stronger guidance first reduces collisions but can eventually displace the policy from task-relevant motions. The regularization sweep and scheduled-versus-constant comparison retain this trade-off; MakeCoffee success drops from 57.2% to 46.2% under constant guidance despite equal 2.8% collision rates.

The PlaceToast default in Tables 7–8 is 71.8% success and 6.9% collisions, whereas Tables 1 and 5 give 74.8% and 4.6%. These source values are retained separately rather than silently reconciled. Cost-gradient scale sweeps also show that collision reduction alone need not preserve precise manipulation.

Complete original figure with every panel, label and legend.
Figure 11: CBF-QP guidance-strength sensitivity analysis. The default γ=1.0\gamma=1.0 follows the linearized QP derivation and provides a robust task–collision tradeoff across the three tasks.
Table 7: CBF-QP regularization sensitivity analysis averaged over the 9 robot embodiments used in the paper. The λcbf=0.01\lambda_{\mathrm{cbf}}=0.01 row corresponds to the default setting.
Task 𝝀𝐜𝐛𝐟\boldsymbol{\lambda_{\mathrm{cbf}}} TSR ↑\uparrow RWD ↑\uparrow COR ↓\downarrow
PlaceToast 0.000 70.8% 0.800 5.9%
0.001 70.0% 0.797 6.1%
0.005 72.4% 0.809 5.0%
0.010 71.8% 0.812 6.9%
0.050 70.9% 0.797 5.3%
0.100 70.1% 0.797 5.7%
0.500 69.9% 0.797 5.9%
1.000 70.9% 0.803 7.1%
TurnOnFaucet 0.000 57.2% 0.829 26.3%
0.001 58.9% 0.842 25.7%
0.005 58.6% 0.833 24.9%
0.010 60.6% 0.833 27.1%
0.050 57.0% 0.817 26.6%
0.100 59.6% 0.831 25.9%
0.500 59.7% 0.838 25.7%
1.000 60.2% 0.835 24.6%
MakeCoffee 0.000 46.3% 0.603 3.6%
0.001 42.4% 0.574 3.2%
0.005 45.3% 0.591 2.6%
0.010 57.2% 0.670 2.8%
0.050 47.2% 0.606 2.0%
0.100 42.7% 0.570 3.4%
0.500 44.7% 0.590 3.6%
1.000 46.4% 0.603 3.6%
Table 8: Guidance schedule ablation averaged over the 9 robot embodiments used in the paper. Scheduled guidance uses Eq. (16); constant guidance keeps γt=γ=1.0\gamma_{t}=\gamma=1.0 for all denoising steps.
Task Guidance scale TSR ↑\uparrow RWD ↑\uparrow COR ↓\downarrow
PlaceToast Scheduled γt\gamma_{t} 71.8% 0.812 6.9%
Constant γt\gamma_{t} 70.2% 0.796 5.6%
TurnOnFaucet Scheduled γt\gamma_{t} 60.6% 0.833 27.1%
Constant γt\gamma_{t} 57.8% 0.827 26.1%
MakeCoffee Scheduled γt\gamma_{t} 57.2% 0.670 2.8%
Constant γt\gamma_{t} 46.2% 0.593 2.8%
Complete original figure with every panel, label and legend.
Figure 12: Cost-gradient guidance-strength sensitivity analysis. Solid curves show the CG baseline under different ρ\rho, while dashed horizontal lines show EmbodiSteer with its default setting. Tuning ρ\rho can reduce collisions, but CG consistently gives a worse task–collision tradeoff.

C.5 Obstacle-Present Training and Obstacle-Estimation Noise

Obstacle-present floating-gripper training raises TurnOnFaucet success only 2.4 percentage points over obstacle-free training and remains 16.6 points below guidance; it still contains no deployed-arm body states.

Geometric estimation noise matters. On MakeCoffee, adding 5/10 mm translation-and-extent standard deviation plus 3° rotation noise lowers success from 57.2% to 50.4%/44.6% and raises collisions from 2.8% to 4.4%/6.8%. The physical evaluation assumes known cuboids, not an implemented unknown-obstacle perception system.

Table 9: Obstacle-estimation noise on MakeCoffee. Translation and cuboid extent share the listed standard deviation; rotation standard deviation is 3∘3^{\circ} for both perturbed settings.
Translation / extent std TSR (%) ↑\uparrow COR (%) ↓\downarrow
No perturbation 57.2 2.8
5 mm 50.4 4.4
10 mm 44.6 6.8

C.6 Demonstration Coverage Settings

One sweep retrains with 20, 50, 100 or 200 demonstrations. Another holds 200 demonstrations and scales motion-planner waypoint randomization to 10%, 33%, 67% or 100%. Those percentages measure diversity settings, not fractions of retained demonstrations; both broader coverage and more examples help guided task completion.

C.7 Candidate Sampling Budget

Increasing candidate count from four to thirty-two raises TurnOnFaucet sampling-baseline success from 43.1% to 45.9%, still below EmbodiSteer’s 60.6%. Latency grows from 1.48 to 10.97 seconds per chunk.

Success is averaged across nine arms on TurnOnFaucet, whereas latency is measured separately on PlaceToast/Panda with an RTX 4090. Table 10 preserves both measurement conditions and timing deviations.

Table 10: Candidate-budget sensitivity of EE w/ Sampling on TurnOnFaucet, averaged over nine robot embodiments, and inference latency (mean ±\pm standard deviation) measured separately on PlaceToast with Panda and an RTX 4090.
Candidates KK 4 8 16 32
TSR (%) 43.1 44.4 43.9 45.9
Latency (s) 1.4803±0.02551.4803\pm 0.0255 2.8291±0.05362.8291\pm 0.0536 5.5529±0.08345.5529\pm 0.0834 10.9728±0.164110.9728\pm 0.1641

C.8 Action-Chunk Continuity

Sixty episodes produce 2,540 paired guided/unguided chunks on Panda and UR5. Each pair shares observation, joints and diffusion randomness, but only the guided chunk executes. The comparison is a trajectory diagnostic, not a matched unguided rollout experiment.

The normalized joint-step 95th percentile changes at most 1.6% in five settings and rises 12.7% on TurnOnFaucet/Panda. Maximum second-difference ratios reach 4.582 on MakeCoffee/UR5 and 2.693 on PlaceToast/UR5, so stable typical steps do not rule out sharper local acceleration changes. Gripper motion is excluded.

Table 11: Continuity over executed action prefixes. P95 step is measured in radians; “ratio” is guided/unguided. r2r_{2} is the maximum second-difference ratio. QP activity is the percentage of active horizon-point constraints.
Task Robot
P95 step
unguided
P95 step
guided
Ratio 𝒓𝟐\boldsymbol{r_{2}}
QP active
(%)
MakeCoffee Panda 0.0348 0.0351 1.011 1.000 10.5
UR5 0.0216 0.0216 0.999 4.582 28.7
TurnOnFaucet Panda 0.0392 0.0442 1.127 1.000 28.8
UR5 0.0135 0.0135 1.003 1.292 54.5
PlaceToast Panda 0.0244 0.0246 1.007 1.000 7.7
UR5 0.0171 0.0174 1.016 2.693 7.2

C.9 Additional Qualitative Results

The full simulation sequences contrast end-effector collisions and arm-link collisions under unguided execution with guided examples. They illustrate observed behavior and do not replace the per-robot failure rates.

Complete original figure with every panel, label and legend.
Figure 13: Additional simulation qualitative results. The base Cartesian policy can collide at the end-effector or along the arm body when obstacles are added at test time, while EmbodiSteer generates collision-aware whole-body motion and completes the task.

Appendix D Real-World Results

The physical appendix retains task definitions, independent success/contact annotations, complete trial counts and every qualitative sequence.

D.1 Setup and Annotation Protocol

MakeIcedCoffee transfers blocks representing ice from a bowl into a cup; PutFlowerInVase inserts a bouquet; ArrangeBanana moves a banana from a pen holder into a cup. Each task uses 200 UMI demonstrations and one checkpoint shared across UR5 and Panda.

Ten trials per robot-task-setting test final object placement without timeout, dropping or human intervention. Obstacles can move when contacted for safety; a robot may push one aside and still complete the task. Any arm or end-effector contact is counted independently as collision.

Complete original figure with every panel, label and legend.
Figure 14: Real-world task protocols. Each row shows representative stages of one UMI-trained task: MakeIcedCoffee, PutFlowerInVase, and ArrangeBanana.

D.2 Full Real-World Results

Table 12 keeps every ten-trial numerator and denominator. Guided outcomes range from 6/10 to 10/10 successes, with zero or one collision per robot-task pair. Totals are 47/60 successes and 4/60 collisions; the small task-level samples should not be read as precise estimates of deployment reliability.

Table 12: Real-world results by robot and task. TSR and COR are reported as success/collision counts over 10 trials for each robot-task pair; the total row aggregates over 60 trials.
Robot Task No Obstacle Obstacle, Base Policy Obstacle, EmbodiSteer
TSR ↑\uparrow TSR ↑\uparrow COR ↓\downarrow TSR ↑\uparrow COR ↓\downarrow
UR5 MakeIcedCoffee 9/10 4/10 10/10 8/10 1/10
PutFlowerInVase 7/10 4/10 9/10 6/10 1/10
ArrangeBanana 8/10 4/10 9/10 9/10 0/10
Panda MakeIcedCoffee 9/10 6/10 10/10 8/10 1/10
PutFlowerInVase 6/10 1/10 10/10 6/10 0/10
ArrangeBanana 10/10 6/10 10/10 10/10 1/10
Total All tasks 49/60 25/60 58/60 47/60 4/60

D.3 Obstacle Layouts and Qualitative Results

Obstacle type and pose vary across episodes. Figures 15 and 17 preserve the complete layouts and both-robot sequences for all three tasks; Figure 16 records representative failures rather than only successful rollouts.

Complete original figure with every panel, label and legend.
Figure 15: Real-world obstacle layouts. The obstacles are highlighted with yellow masks. The obstacle type and pose are varied across episodes.
Complete original figure with every panel, label and legend.
Figure 16: Representative failure modes. Guidance may move the robot into out-of-distribution states, fail to fully prevent collision under local linearization and clipping, or disrupt precision-critical pick/place motions.
Complete original figure with every panel, label and legend.
Figure 17: Real-world qualitative results. From top to bottom: MakeIcedCoffee, PutFlowerInVase, and ArrangeBanana. Each row compares the base Cartesian policy and EmbodiSteer on UR5 and Franka Panda under obstacle layouts that test end-effector and arm-body avoidance.

Appendix E Failure Mode Analysis

Three failure modes remain: corrections can produce observations outside the demonstrations; local, clipped QP steps can leave unavoidable or unresolved contacts; and small gripper deviations can spoil precision-critical grasping or placement. Collision avoidance and task completion remain separate objectives, neither guaranteed by the local guidance rule.

Concept demonstration

Open image ↗