Overview
ALTER adapts a frozen single-arm diffusion policy for two-robot collaboration while trying to preserve solo behavior. This edition retains all four original figures, four tables and nine numbered equations from arXiv v2. The opening framework was restored from the PDF. Results should be read jointly: coordination success measures the new behavior, while source-task success tests whether the same active adapter still permits the old skills.
At a glance
A residual denoising head learns partner-dependent coordination from a small collaborative dataset. Single-agent rollouts from the frozen base policy supply replay without access to its original expert demonstrations. All robots share weights but act from their own camera views, without explicit communication or a solo/team label. On the tested simulation task, ALTER improves collaboration and retains roughly 97% combined source success. In the hardware task it ties mixed-data fine-tuning at 18/20 collaborative successes while preserving more source skills.
I Introduction
Pretrained manipulation policies provide useful component skills, but collaborative demonstrations are costly and adaptation can erase solo competence. ALTER first rolls out the base policy in its source environment, then combines those recorded trajectories with a limited set of team demonstrations to train a correction head. At deployment, the head remains active both alone and in a team. The no-source-data premise concerns the original expert dataset: the method still needs source-environment access to generate replay and visual cues that distinguish solo from collaborative situations.
II Related Work
Diffusion Policy supplies the action-chunk formulation. Multi-agent imitation methods learn collaboration directly, while CoDi guides pretrained models with a specified coordination cost. ALTER instead learns a denoiser correction from demonstrations and rehearses base behavior through generated rollouts. Its additive frozen-backbone design is related to side-tuning and ControlNet. Unlike a runtime expert selector, it receives no external instruction to switch between solo and team policies.
III Preliminaries: Diffusion Models
The method uses the EDM continuous-noise formulation. A denoiser predicts clean action data from Gaussian-corrupted samples, with noise-dependent weighting. Tweedie’s relation converts its output into an estimated score, and the probability-flow differential equation transports a high-noise sample toward an action chunk. The paper keeps the base noise schedule and numerical solver during adaptation; the learned correction changes the vector field evaluated by that solver.
Equations 1–3
| (1) |
| (2) |
| (3) |
IV Problem Formulation
The formal task is a decentralized partially observable process with homogeneous agents, local observation kernels and coupled physical dynamics. Shared policy weights do not provide shared observations: the joint action distribution factors into each robot’s local policy. Each predicts an action chunk, executes a prefix and replans. Collaborative demonstrations record joint trajectories, but training pools separate per-agent observation/action samples. A usable adapter must coordinate when needed while retaining source tasks without a solo/team mode label or the base policy’s original demonstrations. Limited team data and missing expert replay motivate frozen skills plus policy-generated rehearsal.
Equations 4–6
| (4) |
| (5) |
| (6) |
V ALTER
Three choices define ALTER: freeze the pretrained denoiser, add a trainable residual head, and train on both collaborative demonstrations and base-policy rollouts. The same parameters serve solo and collaborative deployment.
V-A Residual Coordination Adapter
The adapter receives the noisy action chunk, noise level, local observation and base denoiser prediction. Adding its output to the base prediction adds the residual divided by noise variance to the score, and adds minus residual divided by noise level to the probability-flow drift. This changes the sampled distribution throughout denoising rather than adding an offset to an already sampled action. A zero residual exactly recovers the base sampler; zero-initializing the output path therefore starts adaptation from the base policy. That identity is conditional on a zero residual, not a guarantee that finite training preserves it everywhere.
Equations 7–8
| (7) |
| (8) |
V-B Mixed-Domain Training
Policy-distilled replay records observations and executed actions from base-policy rollouts in the source environment. On collaborative samples, the complete denoiser minimizes the usual reconstruction loss, with gradients entering only the adapter. On replay samples, the objective instead penalizes the weighted squared residual itself; it does not regress the complete denoiser to the recorded action. This directly encourages the condition that recovers the base distribution. A retention weight balances coordination and rehearsal. Routing is learned from observations rather than an explicit team indicator, and preservation is measured on evaluated source episodes rather than established globally.
Equation 9
| (9) |
V-C Deployment and Implementation Details
Every robot runs the same frozen transformer denoiser plus residual transformer decoder using only its own RGB view. The decoder uses projected base-encoder tokens and a separate lightweight convolutional path over raw images, allowing it to recover partner cues the solo-trained encoder may discard. The original sampler and receding-horizon execution remain in use. The paper describes modest extra inference cost but does not provide a latency benchmark; the largest tested adapter accounts for 29.8% of the combined parameter count.
VI Simulation Experiments
Simulation asks whether limited collaborative data suffice, whether the adapted policy retains solo skills and how adapter capacity changes that tradeoff. Evidence comes from one two-arm collaborative task with randomized layouts, evaluated separately from checkpoint selection.
VI-A Experiment Settings
In TwoArmPlaceWipe, one arm moves a tray aside while the other wipes beneath it, after which both return their objects. Objects, pads and dirt locations vary across demonstrations and tests. Success requires tray x/y error below 1.5 cm, sponge offsets within 8 cm and 6 cm, and at least 60% wipe coverage. Training uses 20, 40 or 60 collaborative demonstrations with equal numbers of distilled solo episodes. Mixed batches contain 256 team and 256 solo samples; team-only training uses 256 samples. AdamW runs 900,000 updates at learning rate 0.0002 and weight decay 0.0001, saving averaged checkpoints every 100,000 steps. Forty episodes select the best coordination checkpoint, which is then evaluated on 200 fresh team episodes and 100 per original solo task. Actions are replanned every 15 environment steps with an 1,800-step cap. The source is inconsistent about pretraining data: this section says 600 expert solo demonstrations, while Table II lists 400 for the base policy; both originals are retained.
VI-B Baselines
All methods use local shoulder-camera images. From Scratch trains on the same mixed demonstrations with parameter count matched to the base-plus-head total. FT-mixed fine-tunes all 17,446,215 base parameters on team and distilled solo data. FT-multi fine-tunes those same parameters using only team demonstrations. Comparing ALTER with FT-mixed tests freezing plus residual adaptation; comparing fine-tuning variants tests mixed replay. These comparisons also differ in architecture and trainable parameter fraction, so no single component alone explains the overall gap.
VI-C Two-Arm Low-Data Coordination
At 20, 40 and 60 collaborative demonstrations, ALTER reaches 35.0%, 71.5% and 87.0% success. From Scratch reaches 13.5%, 28.0% and 59.0%, giving ALTER gains of 21.5, 43.5 and 28.0 percentage points. FT-mixed reaches 33.0%, 59.5% and 63.5%; FT-multi reaches 29.5%, 50.5% and 58.5%. The lowest-data advantage over mixed fine-tuning is only two points, while gains are larger at higher budgets. Table I reports selected-checkpoint episode outcomes rather than confidence intervals or variation across independent training seeds.
| Multi-agent demos | Single-arm demos | Method | Checkpoint | Coordination success |
|---|---|---|---|---|
| 20 | 20 | ALTER | k | 35.0 |
| 20 | 20 | FS | k | 13.5 |
| 20 | 20 | FT-mixed | k | 33.0 |
| 20 | 0 | FT-multi | k | 29.5 |
| 40 | 40 | ALTER | k | 71.5 |
| 40 | 40 | FS | k | 28.0 |
| 40 | 40 | FT-mixed | k | 59.5 |
| 40 | 0 | FT-multi | k | 50.5 |
| 60 | 60 | ALTER | k | 87.0 |
| 60 | 60 | FS | k | 59.0 |
| 60 | 60 | FT-mixed | k | 63.5 |
| 60 | 0 | FT-multi | k | 58.5 |
VI-D Source-Task Retention
With the coordination head still enabled, ALTER retains 97.0%, 97.5% and 97.0% combined source success across budgets, close to the base policy’s 97.0% average of 100% place-return and 94% wipe. From Scratch averages 32.5–61.0%, mixed fine-tuning 46.0–82.5%, and team-only fine-tuning 20.0–34.0%. The worst forgetting occurs on place-return for team-only fine-tuning and wiping for the scratch model. These are 100-episode evaluations per task; they establish retention on this tested pair, not every behavior of the pretrained policy.
| Single-arm demos | Source success | |||||
|---|---|---|---|---|---|---|
| Multi-arm demos | Expert | Distilled rollouts | Method | Place- return | Wipe | Combined |
| 0 | 400 | 0 | Base policy | 100.0 | 94.0 | 97.0 |
| 20 | 0 | 20 | ALTER | 99.0 | 95.0 | 97.0 |
| 20 | 0 | 20 | FS | 56.0 | 9.0 | 32.5 |
| 20 | 0 | 20 | FT-mixed | 55.0 | 37.0 | 46.0 |
| 20 | 0 | 0 | FT-multi | 0.0 | 40.0 | 20.0 |
| 40 | 0 | 40 | ALTER | 100.0 | 95.0 | 97.5 |
| 40 | 0 | 40 | FS | 75.0 | 14.0 | 44.5 |
| 40 | 0 | 40 | FT-mixed | 84.0 | 81.0 | 82.5 |
| 40 | 0 | 0 | FT-multi | 0.0 | 68.0 | 34.0 |
| 60 | 0 | 60 | ALTER | 99.0 | 95.0 | 97.0 |
| 60 | 0 | 60 | FS | 76.0 | 46.0 | 61.0 |
| 60 | 0 | 60 | FT-mixed | 83.0 | 82.0 | 82.5 |
| 60 | 0 | 0 | FT-multi | 0.0 | 50.0 | 25.0 |
VI-E Ablation Studies
With 20 collaborative and 20 distilled solo demonstrations, adapter sizes XS/S/M/L yield 25.0/27.0/33.5/35.0% coordination, while each retains 97.0% combined solo success. Total sizes range from 17.881 to 24.846 million parameters and trainable fractions from 2.4% to 29.8%. Scratch comparators change transformer/feed-forward widths to match total counts within the reported 0.0013%, but perform at 10.0–13.5% coordination. This motivates the largest head for the main experiments, while leaving larger-team scaling and sensitivity to the retention weight untested.
| Size | Method | Total params. (M) | Trainable fraction (%) | Coordination success | Source success |
|---|---|---|---|---|---|
| XS | ALTER | 17.881 | 2.4 | 25.0 | 97.0 |
| XS | FS | 17.881 | 100.0 | 12.0 | 34.5 |
| S | ALTER | 19.077 | 8.5 | 27.0 | 97.0 |
| S | FS | 19.076 | 100.0 | 10.0 | 37.5 |
| M | ALTER | 20.815 | 16.2 | 33.5 | 97.0 |
| M | FS | 20.815 | 100.0 | 10.5 | 39.5 |
| L | ALTER | 24.846 | 29.8 | 35.0 | 97.0 |
| L | FS | 24.846 | 100.0 | 13.5 | 32.5 |
VII Hardware Experiments
A separate hardware study tests the adaptation strategy on a collaborative box task. The base and adapted models are trained on physical demonstrations; this is not direct transfer of the simulation-trained tray/wiping policy.
VII-A Setup
Two xArm7 arms remove a box lid, place a plush bird inside and replace the lid. Success requires the bird inside and a stable unsupported lid covering more than half the box. Base pretraining uses 332 solo demonstrations across lid removal, lid replacement and bird pick-place. Each adaptation method receives 69 collaborative and 68 distilled solo episodes, trains for 25,000 updates and uses only each robot’s shoulder camera. Planning and execution horizons are both 20 steps, with a 900-step trial cutoff. Hardware comparisons include the strongest simulated fine-tuning variant and matched training from scratch.
VII-B Results
ALTER and FT-mixed each succeed in 18/20 collaborative trials, versus 13/20 for From Scratch. ALTER’s source-task results are 19/20 lid removal, 20/20 lid replacement and 20/20 bird pick-place. In Table IV’s column order, From Scratch gives 9/20, 19/20 and 12/20; FT-mixed gives 11/20, 19/20 and 17/20. The narrative paragraph assigns those FT-mixed values to a different task order; this summary follows the labeled table and preserves that discrepancy. The unadapted base is shown at 40/40 on each solo task. With only 20 adapted-policy trials per task, the observed collaboration tie and retention advantage do not establish population-level statistical equality or significance.
VIII Conclusion
Residual denoising plus policy-distilled rehearsal supports both coordination and solo-skill retention in the reported two-arm tasks. The largest simulation coordination gains are 43.5 points over scratch and 23.5 over mixed fine-tuning; the hardware advantage over fine-tuning appears in retained source skills rather than collaborative success. Scope is limited to one simulation task, one physical task, homogeneous two-arm teams and a replay-capable source environment. Broader embodiments, larger teams and diverse task transfer remain future work.