Literature
Display
← All papers

Probabilistic Prioritization of Movement Primitives

Alexandros Paraschos · Rudolf Lioutikov · Jan Peters · Gerhard Neumann

· IEEE Robotics and Automation Letters ↗

Robotics4 min text · CondensedOriginal paper ↗
Contents

At a glance

The method combines learned ProMP controllers through Bayesian task prioritization. Demonstration variability supplies time-varying precision: a task with low variance exerts stronger control than one that permits deviation. A feedback correction improves tracking when composed tasks pull the robot away from its demonstrated states.

Planar simulations and physical iCub and KUKA-arm experiments show adaptation, reuse across task combinations, and addition of an obstacle-avoidance primitive. This is soft prioritization and depends on informative demonstrations; it does not guarantee a strict safety hierarchy.

Figure 1. The iCub reaches with one hand while using the other for supportive contact.
Figure 1. The iCub reaches with one hand while using the other for supportive contact.

I Introduction

Several controllers may compete for the same joints. Fixed priorities can unnecessarily block a task when another only needs accuracy at selected times. Learning time-varying task variability permits compromises that preserve important parts of a movement while allowing useful deviations elsewhere.

III Probabilistic prioritization

The framework first estimates trajectory variability, then uses the resulting stochastic controllers to combine joint-space and operational-space objectives. Learned accuracy becomes the weighting and regularization in the combined control law.

A Encoding accuracy from demonstrations

Gaussian basis weights represent each trajectory, and their fitted distribution determines time-varying state mean and covariance. An operational-space ProMP produces a Gaussian desired acceleration with feedback mean. Its acceleration covariance supplies a task-accuracy measure; low demonstrated variance is interpreted as a need for precision.

Equations 1–3
(1)Original control equation 1.
(2)Original control equation 2.
(3)Original control equation 3.

B Combining tasks

A joint-acceleration distribution acts as the prior. The kinematic relationship between joint and task accelerations supplies a likelihood for the operational-space objective. Bayesian conditioning and marginalization yield a combined Gaussian controller.

The resulting generalized Jacobian inverse is weighted by joint-space covariance and regularized by task-space covariance. Finite task uncertainty permits compromise between objectives; it is not an exact null-space projection.

Equations 4–9
(4)Original control equation 4.
(5)Original control equation 5.
(posterior)Original control equation posterior.
(6)Original control equation 6.
(7)Original control equation 7.
(8)Original control equation 8.
(9)Original control equation 9.

C Multiple tasks

Additional task-space controllers can be incorporated recursively. The paper recommends inserting more important tasks first when numerical checks identify incompatible additions; without such truncation it states that inference order does not change the result. Full-rank task covariance regularizes each inversion.

D Robust distribution tracking

Composition can shift execution outside the distribution for which the original ProMP gains were computed. A likelihood-dependent interpolation adjusts the current mean belief between the demonstration mean and observed state. An extra feedback term then pulls the mean toward its desired evolution. The sigmoid parameters and correction gains remain tunable.

Equations 10–12
(10)Original control equation 10.
(11)Original control equation 11.
(12)Original control equation 12.

IV Relation to optimization

The Bayesian mean corresponds to a soft weighted least-squares trade-off between task constraints and proximity to the joint-space prior. Covariances determine both metrics. The printed objective in Eq. 13 includes an inconsistent argmax label; the described operation minimizes the positive quadratic penalties.

Equation 13
(13)Original control equation 13.

A Relation to strict priorities

As operational-space covariance approaches zero, the method approaches hard prioritization under the usual rank conditions. Choosing identity or inverse-inertia joint covariance recovers familiar generalized inverses. Finite covariance gives a probabilistic interpretation to regularization, allowing lower-priority behavior to influence the solution.

Table I. Conventional, inertia-weighted, and Bayesian generalized inverses.
Table I. Conventional, inertia-weighted, and Bayesian generalized inverses.

V Experiments

Experiments test adapting task-space goals, incorporating expert controllers, pooling demonstrations across task combinations, bimanual reaching, and modifying an existing primitive library. Results combine simulation and two physical robot platforms.

A Data-driven task-space adaptation

Ten planner-generated demonstrations teach a joint-space and a task-space ProMP for a seven-link planar robot. Conditioning the task-space model redirects a via-point while maintaining the demonstrated movement’s overall form. Kinematic reconciliation happens through the controller without a separate inverse-kinematics optimization. The plotted comparison also assesses the added tracking feedback.

Figure 2. Seven-link planar robot before and after task-space conditioning.
Figure 2. Seven-link planar robot before and after task-space conditioning.
Figure 3. Demonstrated, reproduced, conditioned, and corrected endpoint trajectory distributions.
Figure 3. Demonstrated, reproduced, conditioned, and corrected endpoint trajectory distributions.

B Adding external controllers

A thirteen-joint planar robot combines a hip-height trajectory with two hand-designed endpoint attractors. A physical iCub similarly reaches hand targets at different times while holding its hip fixed. These demonstrations show that learned primitives can be composed with supplied controllers.

Figure 4. Physical iCub hand-reaching tasks with a fixed hip.
Figure 4. Physical iCub hand-reaching tasks with a fixed hip.
Figure 5. Planar robot combining two hand targets with hip-height control.
Figure 5. Planar robot combining two hand targets with hip-height control.

C Data efficiency

Two end effectors each reach one of three targets, creating nine combinations while sharing torso joints. Learning each endpoint task independently pools data across combinations; each receives three times the examples available to a separate combination-specific model. Over ten trials, this improves final-position squared-error reward as demonstrations per combination vary.

Figure 6. Nine combinations of three left- and three right-endpoint targets.
Figure 6. Nine combinations of three left- and three right-endpoint targets.
Figure 7. Reward versus demonstrations for pooled task primitives and separate combination-specific models.
Figure 7. Reward versus demonstrations for pooled task primitives and separate combination-specific models.

D Support contacts during reaching

The iCub reaches toward one of three objects while its other hand makes one of three supportive contacts. Ten demonstrations teach each reach and each support location separately. The resulting primitives reproduce and recombine these behaviors. Table II reports centimeter-scale endpoint errors for all nine pairs; it does not provide a general balance guarantee.

Table II. Endpoint errors for the nine supportive-contact and object-reaching combinations.
Table II. Endpoint errors for the nine supportive-contact and object-reaching combinations.

E Adapting a primitive library

Three physical-arm skills—bottle approach, peg placement, and button pressing—use ten demonstrations each and generalize within a 25×25-centimeter area. A newly placed obstacle causes collisions. An additional demonstrated avoidance primitive is combined with each original skill without retraining the library.

Across ten reproductions per case, avoidance rates improve from 0/0.1/0.3 to 0.9/1/1. Endpoint errors after adaptation are 0.48, 0.67, and 0.45 centimeters on average. Standard prioritization avoids the obstacle but has larger endpoint errors. The robot does not perceive the obstacle: its fixed location is encoded by the added demonstration.

Figure 8. Physical primitive library and the newly introduced obstacle.
Figure 8. Physical primitive library and the newly introduced obstacle.
Figure 9. Original colliding trajectories and their adapted obstacle-avoiding counterparts.
Figure 9. Original colliding trajectories and their adapted obstacle-avoiding counterparts.
Table III. Obstacle avoidance and endpoint error across ten reproductions per case.
Table III. Obstacle avoidance and endpoint error across ten reproductions per case.

VI Conclusion

Learning uncertainty as relative priority supports reusable task-space movements and data sharing across combinations. The approach reproduces the provided demonstrations, so poor examples can encode behavior inconsistent with the user’s intent. More complex physical interactions remain future work.

Appendix: including robot dynamics

Rigid-body inverse dynamics maps the combined joint-acceleration Gaussian into a torque distribution using inertia, Coriolis/centripetal terms, and gravity. The resulting mean connects the probabilistic controller with resolved-acceleration control; the original unnumbered expression and Eq. 14 are retained.

Equations torque–14
(torque)Original control equation torque.
(14)Original control equation 14.
Concept demonstration

Open image ↗