At a glance
RMP² evaluates the same structured motion policies as RMPflow through automatic differentiation instead of a separately implemented message-passing tree. It supports general acyclic task graphs, differentiable policy learning, and lower memory use under the stated complexity assumptions. Simulated reaching experiments show how learned residual task policies can improve performance while retaining useful collision-avoidance structure. The contribution is an implementation and learning interface, not a new policy class or unconditional safety guarantee.
I Introduction
Analytic controllers provide structure and conditional guarantees but can be difficult to tune; unrestricted learned policies can require extensive interaction and fail outside their training conditions. RMPflow combines task-specific acceleration policies through geometric maps and state-dependent importance matrices. RMP² makes this composition easier to differentiate and optimize, allowing hand-designed collision avoidance or joint-limit policies to coexist with learned components. The resulting policy is mathematically the same as the corresponding RMPflow policy. [6] [30]
| Algorithm | Time | Space | Req. Tree | Req. Auto.Diff. |
|---|---|---|---|---|
| RMP2 (Algorithm 3) | ✗ | ✓ | ||
| Naïve Implementation (Algorithm 4) | ✗ | ✓ | ||
| RMPflow (Algorithm 1) [6] | ✓ | ✗aafootnotemark: a |
II Background
II-A Acceleration-based Motion Control
The robot’s state consists of configuration and velocity; its policy outputs acceleration, which a low-level controller tracks. Different objectives are naturally expressed in different coordinates, such as end-effector position or distance to an obstacle. Because these coordinates share the same robot configuration, independently designed task policies cannot generally be combined by simply averaging their actions.
II-B RMPflow: A Framework for Multi-Task Problems
Each Riemannian motion policy specifies a desired task acceleration and a positive-semidefinite importance matrix. RMPflow propagates states down a task tree, then combines forces and metrics back toward the configuration space. Its result solves a weighted least-squares problem subject to kinematic consistency. Stability follows only when the task policies and metrics satisfy the framework’s required conditions; arbitrary learned components do not inherit it automatically.
Equations 1–2
| (1) | ||||
| s.t. | (2) | |||
III RMP 2 based on Automatic Differentiation
RMP² computes the configuration-space solution using standard differentiation operators on the task-map computation graph. The retained pseudocode gives both the original message-passing method and its automatic-differentiation replacement.
III-A Key Idea
Eliminating intermediate kinematic constraints converts the tree problem into an unconstrained weighted least-squares problem in root acceleration. Its closed-form solution aggregates leaf Jacobians, metrics, desired accelerations, and curvature corrections. Efficiently evaluating this expression is the central computational challenge.
Equations 3–4
| (3) |
| (4) |
III-B RMP 2 based on Reverse Accumulation
Reverse accumulation computes Jacobian–vector products without explicitly storing every full Jacobian. Forward evaluation supplies task positions, velocities, and curvature terms; leaf policies supply local forces and metrics. Auxiliary copies of the root coordinates then let gradient and Jacobian operations recover the combined root metric and force. A pseudoinverse resolves the final acceleration. Algorithms 2 and 3 preserve the precise differentiation construction.
Equation
Equations 5–6
| (5) |
| (6) |
III-C Complexity of RMP 2
With N graph nodes, L leaves, maximum branching factor b, and node dimension bounded by d, the analysis gives time O(Nbd³) and memory O(Nd + Ld²). The original RMPflow implementation has the same asymptotic time but O(Nd² + Ld²) memory under the paper’s assumptions. These bounds depend on the costs assumed for maps, leaf policies, and differentiation.
III-D Discussion: A Naïve Alternative Algorithm
The direct alternative explicitly constructs each root-to-leaf Jacobian, introducing an additional factor of L in time and large intermediate differentiation graphs. Replacing its curvature computation with Jacobian–vector products reduces memory to the RMP² bound but leaves the extra time factor. Algorithm 4 retains this comparison implementation.
III-E Key Benefits of RMP 2
The practical gains are a simpler task-map interface, support for directed acyclic graphs, ordinary differentiation through learnable maps and policies, and reduced memory use. A tree can represent the same relationships, but constructing it may introduce larger intermediate states or repeated calculations.
IV RMP 2 for Learning
Structured learning can optimize either individual task policies or their coordinate maps. The framework also permits fixed controllers to supply prior behavior while learned residuals improve a selected objective.
IV-A Parameterizing RMP 2 Policies
Leaf accelerations and importance matrices can be parameterized by neural networks; suitable parameterizations preserve properties such as positive semidefiniteness. Task maps can also be learned so that control becomes simpler in a latent space. The choices determine which theoretical properties remain available.
IV-B Learning Setups
Behavior cloning differentiates the complete policy to match expert actions. In reinforcement learning, RMP² can be part of the differentiable policy or treated as an environment-side transformation of a learned leaf action. The latter avoids differentiating the composition and is convenient for a small learned task space; learning many leaves may favor the former.
IV-C Learning with Residuals
An unrestricted residual can be added directly to the final robot acceleration, or a residual task policy can be inserted before composition. The second choice preserves the controller’s task structure and allows fixed obstacle-avoidance policies to mediate the learned change. Initialization near a useful hand-designed policy improves early behavior in the reported experiments.
V Experiments


V-A Three-Link Robot Reaching
A planar three-link arm reaches random goals while avoiding one or three cylindrical obstacles. Three progressively harder settings compare a neural policy from scratch, a residual on the final acceleration, and a residual end-effector RMP. All use PPO; the plots summarize four random seeds over 500 training iterations. The RMP residual parameterizes its metric through a matrix factor so it remains positive-semidefinite. [34]
All methods learn the easiest task, but the unrestricted neural policy collides frequently during training. With more obstacles, high reward can coexist with unsafe behavior. In the hardest goal distribution, the structured residual learns more effectively than either neural alternative. These results demonstrate an empirical benefit from the retained controller structure, rather than proving collision freedom for arbitrary tasks, dynamics, or learned policies.
V-B Franka Robot Reaching
The second experiment uses a simulated seven-degree-of-freedom Franka arm, three spherical obstacles, and goals at least 0.5 m from the initial end effector. Both residual methods outperform learning from scratch. The unrestricted residual obtains slightly higher reward, while the RMP residual records no collisions during these training runs. The evaluation is simulation-only, and zero observed collisions is not a universal safety certificate.
Appendix A The RMPflow Algorithm
RMPflow stores state and a force–metric pair at each tree node. Pushforward applies task maps and Jacobians to positions and velocities; pullback combines child metrics and curvature-corrected forces; resolve applies the metric pseudoinverse at the root. The retained equations give the exact transformations used by Algorithm 1.
Equation 7
| (7) |
Appendix B Complexity Analysis of RMP 2
The complexity proof accounts separately for task-map evaluation, double-backward Jacobian–vector products, leaf evaluation, root differentiation, and the final solve. Reverse-mode differentiation retains O(Nd) node values, while leaf matrices require O(Ld²) storage. Repeated scalar gradient calls dominate the O(Nbd³) time bound; intermediate graphs need not all remain stored once further derivatives are unnecessary.
Appendix C Complexity of RMPflow
For the comparison with RMPflow, graph parents are grouped into tree nodes and duplicated values can share memory. Pushforward and leaf evaluation have comparable costs, but pullback stores matrix-valued quantities throughout the tree. This produces the additional O(Nd²) memory term while retaining O(Nbd³) time.
Appendix D Experimental Validation of Time Complexity
A synthetic chain graph varies in length from four to 36 while node dimension and branching factor stay at three. Timing averages 1,000 evaluations of a TensorFlow static graph and excludes compilation. RMP² scales approximately linearly with graph size; explicitly constructing Jacobians grows superlinearly. The test supports the stated scaling comparison, not a hardware-independent runtime guarantee.
Appendix E RMP 2 versus RMPflow: A Case Study
A three-link arm with nine collision-control points illustrates the interface difference. RMP² uses the natural computation graph of joint angles, link poses, and interpolated points. An explicit RMP tree must package sufficient parent information and can repeat upstream kinematics. Adding pairwise self-collision relationships is straightforward in the graph representation but can require redesigning the tree. Careful tree implementations can reduce redundant work; the comparison emphasizes programming and representation costs.