At a glance
CCDP modifies diffusion-policy sampling after a failed attempt, steering the robot toward alternative demonstrated behaviors. Separate models capture actions, current state, short history, and avoidance of an individual failure; composing the avoidance model repeatedly handles a variable failure memory. Recovery training data are synthesized from successful demonstrations.
Experiments compare five tasks with unknown or changing action feasibility. The method needs a failure detector, manually chosen failure features, and an available alternative behavior; it assumes the reason a remembered attempt failed stays relevant.
Introduction
Repeatedly sampling a policy can repeat the same unsuccessful behavior. CCDP instead remembers failed regions of the action space and guides subsequent samples away from them, using successful demonstrations as the source of alternatives.
A long history need not be encoded in a single network: a product of conditional models combines evidence from the current state, recent motion, and each remembered failure. This avoids training on every ordering of failures, but the assumed cause of failure is static. A dissimilar action is a recovery candidate, not guaranteed to succeed.
Related Works
Generative imitation methods model many valid behaviors, while products of experts combine constraints or preferences expressed by different models. CCDP applies these ideas to low-level failure recovery. Unlike selecting one labeled action region through a high-level planner, its guidance excludes failed regions while leaving other possibilities available. [2] [16]
Training uses successful demonstrations rather than explicit failure-recovery demonstrations or simulator exploration. This does not remove the need for detecting failures during execution or for selecting an informative failure representation.
Preleminaries
The base model predicts demonstration actions by reversing a learned noise process.
Diffusion Policy
A denoising model learns to predict added noise from an action and its observation/action history. At inference it starts from Gaussian noise and repeatedly denoises to generate an action or short trajectory. The original training loss and sampling update are retained below.
Equations
Methodology
The method separates ordinary action generation from the effect of state, history, and remembered failures.
Problem Definition
The policy conditions on current state, a short history, and a set of failure features. A supplied feature function summarizes each failed action and state—for example, its action vector, final position, or primitive label. Directly feeding every failure to one network would make input size and training coverage grow with memory length.
Equation 1
| (1) |
Composition of Diffusion Policies
CCDP combines an unconditional action model with state-, history-, and failure-conditioned models. Their weighted denoising corrections are added at each sampling step. The state term encourages locally appropriate actions; the history term encourages continuity; each avoidance term suppresses an already unsuccessful region.
Only one failure-conditioned model is learned and reused for all remembered failures. Positive guidance weights need not sum to one or be below one. Reducing the history weight after a failure helps the controller leave a bad trajectory; using only the history-conditioned term recovers the standard diffusion-policy case.
Equations 2–4
| (2) |
| (3) |
| (4) |
Recovery Model
Recovery candidates are pairs of sufficiently different actions at sufficiently similar states, measured using chosen feature-space and state-space thresholds. The method samples extra actions around demonstrated states using the unconditional and state-conditioned models, deliberately omitting history to increase diversity. Accepted pairs then train the avoidance-conditioned denoiser.
This synthetic data encode alternatives, not observed successful recoveries. Their usefulness depends on the demonstration repertoire and on the assumption that repeating a similar action at a similar state would repeat the failure.
Equations 5–7
| (5) |
| (6) |
| (7) |
1 Failure Key Features
Three feature choices are illustrated: the failed action itself, its final state, or a discrete action-primitive label. The first two can avoid manual class labels; the primitive-based experiment does require them. During execution, detected failures are added to the memory and influence the next composed sample. Learning the features automatically is future work.
Experiments
Evaluation uses 100 randomized scenarios per task. Demonstrations are generated in simulation by a planner with true task parameters, which are hidden from the learned controllers. Policies train from those demonstrations; simulation is therefore used to collect data even though policy fitting does not use a dynamics model.
Baselines are ordinary DP and DP*, which rejects failed predefined regions. Tests report both task completion and how well an implicit preference in the demonstrations is followed. Table I records history, prediction, and execution lengths; denoising uses 100 steps. The available evidence is these controlled tasks, rather than unrestricted physical failure recovery.
Door Opening (DO)
The robot must open a door whose direction—such as sliding, lifting, or pulling—is unknown. Failed actions provide avoidance features. Success requires opening within five attempts. CCDP is comparable to the region-labeled DP* baseline and improves over ordinary DP without needing door-direction labels.
Button Pressing (BP)
The button location is unknown within a search area; success requires finding it within 30 attempts. CCDP avoids failed target positions continuously. DP* instead removes predefined bins after failures, and is less efficient in the reported comparison.
Object Manipulation (OM)
Object mass is hidden. Demonstrations contain single-arm lifting, bimanual lifting, and pushing with frequencies 70%, 20%, and 10%. The intended preference is to use the least demanding feasible option. Here failure features are primitive labels.
CCDP and DP* improve task success over DP, with DP* slightly ahead in completion. Preference adherence differs: DP reaches 88%, CCDP 66%, and DP* 38%. Thus recovery success and fidelity to the demonstrated preference are separate outcomes; CCDP does not dominate every metric.
Objects Packing (OP)
Twelve objects must fill three baskets of capacity four. Completed or full destinations become regions to avoid, and demonstrations favor nearer baskets. CCDP and DP* both exceed 90% task success; CCDP more often chooses the closest available basket. Final trajectory states supply avoidance features.
Bartender (BT)
The bartender task requires filling every cup, ideally choosing the closest unfilled cup next. Once a cup is full, its destination is excluded from future sampling. This extends the same mechanism from remembered failures to remembering completed subgoals.
Discussion
The comparison separates completing the task from respecting preferences encoded by demonstration frequency.
CCDP vs DP ∗
DP* commits the sampler to one remaining region. CCDP excludes a failed region while allowing state and history information to guide choices elsewhere. This explains its flexibility, although it does not eliminate manual design of failure features or all labeling in the experiments.
NOT Operation
The authors also tried learning neighborhoods around failed actions and applying distribution negation, analogous to NOT. These attempts were unstable. The reported approach instead learns avoidance distributions directly and combines them through products.
Limitations and Future Work
Failure features and guidance weights are selected manually. Larger weights can encourage exploration but also destabilize sampling. Future work includes learned features, automatic weight adjustment, and better recovery-data generation. The method can only find useful alternatives supported by its learned distributions and requires reliable failure detection.
Conclusion
Composing conditional diffusion models provides a low-level mechanism for moving away from failed or completed actions. The experiments show better recovery than repeated ordinary sampling and a useful trade-off against region-based rejection. The results depend on the chosen features, failure memory, guidance weights, and demonstration coverage.