Literature
Display
← All papers

End-to-End Stable Imitation Learning via Autonomous Neural Dynamic Policies

Dionis Totsila · Konstantinos Chatzilygeroudis · Denis Hadjivelichkov · Valerio Modugno · Ioannis Hatzilygeroudis · Dimitrios Kanoulas

Presented · ICRA 2023 L3H2 Workshop ↗

Robotics3 min text · CondensedOriginal paper ↗
Contents

At a glance

Autonomous Neural Dynamic Policies (ANDPs) combine stable linear dynamical systems using positive neural-network weights. The resulting policy can condition on images while driving its controllable state to a fixed attractor. Simulated multitask motions and a physical pouring experiment demonstrate imitation and perturbation recovery. The convergence guarantee assumes commanded velocities are followed; it is not a guarantee for the complete robot–environment system.

I Introduction

The aim is to retain neural-network flexibility while giving movement policies an analyzable convergence property. ANDPs separate directly controlled coordinates from contextual observations and use the latter to select a state-dependent mixture of stable dynamics. Their differentiable parameterization permits unconstrained gradient-based fitting.

Refer to caption

Fig. 1: Autonomous Neural Dynamic Policy Outline.

II Related Work

Dynamic Movement Primitives provide structured movement generation, while SEDS and LPV-DS learn autonomous stable vector fields. Neural Dynamic Policies combine neural predictions with movement primitives but do not provide this paper’s fixed-attractor guarantee. ANDPs replace Gaussian mixture gating with neural gating while constraining each elementary system. [7], [17], [18].

III Problem formulation

Training receives demonstrated state sequences rather than expert control commands. A policy should reproduce those trajectories under unknown system dynamics. A low-level controller may be needed to convert desired state velocities into executable robot inputs.

Equation 1
𝒙t+1=f⁡(𝒙t,𝒖t)+𝒘\displaystyle\boldsymbol{x}^{t+1}=f(\boldsymbol{x}^{t},\boldsymbol{u}^{t})+\boldsymbol{w} (1)

IV Proposed Policy Structure

The policy is a positively weighted sum of linear systems sharing one attractor. Positive-definite symmetric parts of their matrices yield a common Lyapunov decrease and asymptotic convergence of the directly controlled state. Contextual observations can change the weights without changing the attractor. The theorem assumes perfect velocity tracking and does not establish collision avoidance, bounded commands, or stability of uncontrolled environment variables.

Equations 2–4
𝒙=[𝒙c𝒙n​c]∈ℝdc+dn​c,\displaystyle\boldsymbol{x}=\begin{bmatrix}\boldsymbol{x}_{c}\\ \boldsymbol{x}_{nc}\end{bmatrix}\in\mathbb{R}^{d_{c}+d_{nc}}, (2)
𝒙˙c=π⁡(𝒙)=∑i=1Nwi​(𝒙)​𝑨i​(𝒙c∗−𝒙c)\dot{\boldsymbol{x}}_{c}=\pi(\boldsymbol{x})=\sum_{i=1}^{N}w_{i}(\boldsymbol{x})\boldsymbol{A}_{i}\Big(\boldsymbol{x}_{c}^{*}-\boldsymbol{x}_{c}\Big) (3)
{𝑨i+𝑨iT≻0the symmetric part of ​A​ is psdwi​(𝒙)>0,i=1,..,N,∀𝒙∈ℝE\begin{cases}\boldsymbol{A}_{i}+\boldsymbol{A}_{i}^{T}\succ 0&\text{the symmetric part of }A\text{ is psd}\\ w_{i}(\boldsymbol{x})>0,&i=1,..,N,\forall\boldsymbol{x}\in\mathbb{R}^{E}\end{cases} (4)

IV-A ANDPs via Neural Networks

A neural network predicts softmax mixture weights. Each system matrix is represented by a positive-definite component plus a skew-symmetric component; a Cholesky factor with positive diagonal ensures the former condition. These substitutions enforce stability during unconstrained optimization. Experiments fix the attractor to the average final demonstrated state.

V Experiments

The evaluation uses simulated image-conditioned motions and a physical Franka Panda pouring task. It tests trajectory reproduction, switching task cues during execution, recovery from external forces, and end-effector orientation control. The experiments use imitation learning; reinforcement learning is suggested rather than evaluated.

Refer to caption

Fig. 2: Multi-task scenario with image inputs. All tasks are learned with a single model that can distinguish between tasks given an image input.

V-A Multi-Task Learning

One demonstration each of linear, sinusoidal, and curved motion trains a single policy. A grayscale 64×64 image selects the task through a convolutional gating network. Changing the image at one second switches the motion pattern; force perturbations at zero and five seconds also produce recovery toward the shared attractor. These are illustrative rollouts rather than a broad quantitative robustness benchmark.

Refer to caption

Fig. 3: Reactiveness of ANDPs on changes in the non-controllable part of the state, we switch from the line image to the sine image at t=1​s1s.

Refer to caption

Fig. 4: Reactiveness of ANDPs on external force perturbations we apply an external force twice, one at t=0​st=0s and t=5​st=5s.

V-B Physical Robot Experiment

Three kinesthetic demonstrations teach a six-coordinate end-effector pouring motion, using plastic pieces to emulate liquid. Across ten initial robot configurations, the median fraction of pieces reaching the bowl is 100%, with the 25th and 75th percentiles at 67.5% and 100%. This metric measures transported pieces, not the fraction of fully successful trials.

Refer to caption

Fig. 5: Collecting pouring task demonstrations via kinesthetic guidance on the Franka Emika Panda robot.
Original figure
Fig. 6: Percentage of objects that ended up inside the bowl on the 10 replications of the pouring task.
Refer to caption
Refer to caption
Refer to caption
Refer to caption
Fig. 7: From left to right, screenshots of a successful trial of the pouring task in the physical setting.

VI Results and Conclusions

The architecture supports image and pose inputs with a fixed-attractor convergence guarantee under its tracking assumption. A shared fixed attractor limits long-horizon tasks and nonmonotonic motions; moving goals, reinforcement learning, and broader image-based evaluation remain future work.

Acknowledgments

The authors acknowledge support from the Hellenic Foundation for Research and Innovation and the UK Research and Innovation Future Leaders Fellowship.

Concept demonstration

Open image ↗