Literature
Display
← All papers

Robustness without Wrinkles: Parallel Simulation and Robust MPC for Certified Deformable Manipulation

Wei-Chen Li · Jeffrey Fang · Sasanka Polisetti · Yuexi Song · Glen Chou

Preprint posted · CoRL 2026 (accepted) ↗

Robotics10 min text · CondensedOriginal paper ↗
Contents

Reading guide

All twenty-one pages, nine complete figures, three tables, seventeen numbered equation groups, two proof displays, both theorem statements, the full proof and sixty-one references are covered. CORD-SLS combines a contact-aware simulator with robust output-feedback planning; analytical policy gradients are a separate learning experiment. Its safety statements require valid finite-horizon tubes, calibrated disturbances and bounded state-estimation error. The hardware's fixed 3 cm radius with 87% calibration coverage does not itself establish a specified trajectory-level confidence guarantee.

Complete original Figure 1: Hardware results showcase CORD-SLS’s real-world efficacy. In a) , CORD-SLS robustly solves a constrained rope manipulation task, lifting the rope and navigating over an obstacle with our contact-smoothed differentiable simulator. In b) , we show the associated robust tubes computed by CORD-SLS  when avoiding the obstacle (red). In c) , CORD-SLS extends to cloth folding.
Figure 1: Hardware results showcase CORD-SLS’s real-world efficacy. In a), CORD-SLS robustly solves a constrained rope manipulation task, lifting the rope and navigating over an obstacle with our contact-smoothed differentiable simulator. In b), we show the associated robust tubes computed by CORD-SLS  when avoiding the obstacle (red). In c), CORD-SLS extends to cloth folding.
Website Code Video
Original project resource badges: website, code and overview video.

At a glance

A GPU-parallel simulator supplies contact-seeking surrogate gradients while preserving nonsmooth forward rollouts. Parallel system-level synthesis propagates model and measurement uncertainty to tighten constraints. The experiments compare rope and cloth planning, policy learning and a small hardware demonstration. Main-table planning takes 50–70 ms for rope and 711–758 ms for cloth, while policy inference is faster. The useful result is efficient contact-aware planning with explicitly conditional robustness, rather than an unconditional safety certificate for every deployed perception and replanning pipeline.

1 Introduction

Rope and cloth have many coordinates, intermittent contact and uncertain state estimates. Nominal plans may fail when these uncertainties are ignored, while sparse-contact RL is difficult to explore. The framework combines differentiable contact discovery, output-feedback reachable tubes and aggregate residual calibration. Each part addresses a different problem: the simulator supports optimization, SLS bounds specified disturbances, and conformal prediction supplies a statistical radius under its stated data assumptions.

2 Related Work

Related work spans deformable simulation, differentiable contact, learned/geometric planners and robust control. CORD-SLS integrates these components rather than replacing physical modeling or state estimation. Runtime comparisons depend on simulator representation, compilation and hardware; a faster quasi-static model does not imply greater fidelity to transient deformation.

Deformable Object Simulation

Position-based methods trade fidelity for tractability, while rod/shell models represent richer bending, twisting and friction. The paper uses a convex contact relaxation and surrogate smoothing to obtain useful optimization directions. DEFORM, DaXBench and smoothing-error reachable tubes are relevant comparisons. The chosen elastic model captures stretch and bending; it does not explicitly retain all twisting degrees of freedom of the rod baseline.

Planning

The method extends GPU-SLS from state to output feedback, drawing on Vision-SLS. Prior abstractions and learned models can plan useful motions but do not automatically bound contact and perception error. The new integration still depends on a known state representation, task costs/constraints and valid uncertainty assumptions. Recursive feasibility under repeated replanning is a separate issue discussed in AppendixD.

3 Problem Statement

Equation 1 models bounded process disturbances and delayed noisy observations, and Equation 2 defines the planner's simulator. The state includes deformable nodes and actuated coordinates. The desired output-feedback controller may use observation history, while safety is expressed over state/control constraints. The original HTML entirely omits Equation 1 a/b; both equations are restored from the PDF with selectable mathematical text. Full-state visual reconstruction in the hardware implementation is an additional modeling choice, not direct ground-truth state access.

Original mathematical definitions, assumptions and evaluation protocol

We consider a known state representation q∈𝒬⊆ℝnqq\in\mathcal{Q}\subseteq\mathbb{R}^{n_{q}} for a deformable object of interest (e.g., 3D positions of links for rope, or of mesh node positions for cloth) actuated by control input u∈𝒰⊆ℝnuu\in\mathcal{U}\subseteq\mathbb{R}^{n_{u}} and governed by uncertain dynamics (1a), where f:𝒬×𝒰→𝒬f:\mathcal{Q}\times\mathcal{U}\rightarrow\mathcal{Q}. To account for perception latency, we model the system as being observed with uncertain time-delayed measurements y∈𝒴⊆ℝnyy\in\mathcal{Y}\subseteq\mathbb{R}^{n_{y}} according to an observation map h:𝒬→𝒴h:\mathcal{Q}\rightarrow\mathcal{Y} (1b),

Complete original mathematical display S3.E1
Original selectable mathematical display
Equations 1a–1b
qk+1=f(qk,uk)+E(qk)wkq_{k+1}=f(q_k,u_k)+E(q_k)w_k(1a)
yk+1=h(qk)+F(qk)eky_{k+1}=h(q_k)+F(q_k)e_k(1b)
Original mathematical definitions, assumptions and evaluation protocol

where E:𝒬→ℝnq×nqE:\mathcal{Q}\rightarrow\mathbb{R}^{n_{q}\times n_{q}} and F:𝒬→ℝny×nyF:\mathcal{Q}\rightarrow\mathbb{R}^{n_{y}\times n_{y}} and w∈ℬnq:={w∈ℝnq∣‖w‖2≤1}w\in\mathcal{B}^{n_{q}}:=\{w\in\mathbb{R}^{n_{q}}\mid\|w\|_{2}\leq 1\} and e∈ℬnye\in\mathcal{B}^{n_{y}} are bounded disturbances. We aim to design 1) a differentiable deformable object simulator that approximates the dynamics (1a) while remaining friendly for planning and learning, and 2) an efficient and robust planning and control algorithm that uses this simulator to solve reachability-aware MPC problems for safe deformable object manipulation. We consider two specific problems:

Complete original mathematical display S3.E2
Original selectable mathematical display
Equation 2
qk+1=f⁡(qk,uk).q_{k+1}=f(q_{k},u_{k}). (2)
Original mathematical definitions, assumptions and evaluation protocol

Problem 2. Reachability-informed MPC for robust deformable manipulation. Optimize a nominal length-NN state z:={zk}k=0N\textbf{z}:=\{z_{k}\}_{k=0}^{N} and control v:={vk}k=0N−1\textbf{v}:=\{v_{k}\}_{k=0}^{N-1} trajectory that is feasible for the simulator dynamics (2), together with a causal output-feedback controller π:=(π0,…,πN−1)\pi:=(\pi_{0},\ldots,\pi_{N-1}), where πt:𝒴t+1→𝒰\pi_{t}:\mathcal{Y}^{t+1}\rightarrow\mathcal{U} stabilizes the true deformable dynamics (1a) about (z,v)(\textbf{z},\textbf{v}) using the history of observations, and ensures closed-loop satisfaction of state and control constraints 𝒮⊆𝒬×𝒰\mathcal{S}\subseteq\mathcal{Q}\times\mathcal{U}.

4 Method

The approach has four components: implicit deformable simulation, GPU-parallel output-feedback SLS, analytical policy gradients and conformal residual calibration. Their guarantees and timings should be assessed separately. Smoothing improves optimization directions, but a surrogate gradient is not the exact derivative of the nonsmooth rollout. A calibrated one-step residual radius is not automatically an infinite-horizon or recursively feasible control guarantee.

4.1 Differentiable Deformable Object Simulator with Contact Smoothing

Equations 3–7 define a quasi-static nodal model, stiff holonomic attachments and a two-stage implicit/contact solve. Convex friction-cone constraints approximate contact; they do not make every nonlinear elastic problem globally convex. Equations 8–10 differentiate implicit/KKT systems, reusing forward factorizations. Backward solves and vector–Jacobian products still have computational cost: the claim of no extra cost should be read as avoiding new factorization, not free differentiation. Equations 3/6 print M divided by the squared time step, whereas Equation 7 prints M in its Hessian; the original convention is retained and needs reconciliation when reproducing the implementation. Contact smoothing biases gradients only; forward predictions remain nonsmooth. It helps discover contact without guaranteeing every gradient step solves the physical task.

Original mathematical definitions, assumptions and evaluation protocol

Consider a nodal system with nvn_{v} nodes, whose 3D coordinates, together with the nun_{u} actuated degrees of freedom (DoFs), form the configuration vector q∈ℝnqq\in\mathbb{R}^{n_{q}}, where nq=3​nv+nun_{q}=3n_{v}+n_{u}. For an elastic energy E⁡(q)E(q), the corresponding internal elastic force is given by −∂E(q)/∂q-\partial E(q)/\partial q. Under a quasi-static assumption, the system satisfies

Complete original mathematical display S4.E3
Original selectable mathematical display
Equation 3
M​q+−qδ​t2+∂E⁡(q+)∂q+=τ⁡(u)+∑i=1ncJi​(q)⊤​λi,M\frac{q^{+}-q}{\delta t^{2}}+\frac{\partial E(q^{+})}{\partial q^{+}}=\tau(u)+\sum_{i=1}^{n_{c}}J_{i}(q)^{\top}\,\lambda_{i}, (3)
Original mathematical definitions, assumptions and evaluation protocol

where MM is the mass matrix, τ⁡(u)\tau(u) are the external generalized forces, including gravity acting on the nodal system and actuation forces applied to the actuated DoFs, λi∈ℝ3\lambda_{i}\in\mathbb{R}^{3} is the iith constraint force, δ​t\delta t is the discrete timestep, and Ji∈ℝ3×nqJ_{i}\in\mathbb{R}^{3\times n_{q}} is the corresponding constraint Jacobian. Both the elastic and constraint forces are evaluated at the next time step, yielding an implicit time-stepping scheme for improved numerical stability. For holonomic constraints, such as a gripper rigidly interacting with the nodal system, we suppose an active constraint ci​(q)=0∈ℝ3c_{i}(q)=0\in\mathbb{R}^{3} enforces the gripper attachment. The next configuration q+q^{+} and constraint force λi\lambda_{i} must satisfy the complementarity condition

Complete original mathematical display S4.E4
Original selectable mathematical display
Equation 4
0=Ji​(q)​(q+−q)+ci​(q)⟂λi∈ℝ3,0=J_{i}(q)\,(q^{+}-q)+c_{i}(q)\;\perp\;\lambda_{i}\in\mathbb{R}^{3}, (4)
Original mathematical definitions, assumptions and evaluation protocol

where the constraint Jacobian is defined as Ji​(q):=∂ci​(q)/∂qJ_{i}(q):=\partial c_{i}(q)/\partial q. For contact constraints, we adopt Anitescu’s convex relaxation [53]. Here, the next step configuration q+q^{+} and contact force λi\lambda_{i} satisfy

Complete original mathematical display S4.E5
Original selectable mathematical display
Equation 5
ℱμ∗∋Ji​(q)​(q+−q)+[ϕi​(q)00]⊤⟂λi∈ℱμ,\mathcal{F}_{\mu}^{*}\ni J_{i}(q)\,(q^{+}-q)+\begin{bmatrix}\phi_{i}(q)&0&0\end{bmatrix}^{\top}\;\perp\;\lambda_{i}\in\mathcal{F}_{\mu}, (5)
Original mathematical definitions, assumptions and evaluation protocol

where ϕi​(q)\phi_{i}(q) is the signed distance of contact pair ii, ℱμ\mathcal{F}_{\mu} is the friction cone, and ℱμ∗\mathcal{F}_{\mu}^{*} is its dual cone.

Original mathematical definitions, assumptions and evaluation protocol

Simulation Solver We solve the dynamics in (3) subject to the holonomic and contact constraints in (4) and (5) using a two-stage procedure. The first stage computes the system response under the holonomic constraints, while the second stage resolves contact interactions. In the first stage, we solve for an intermediate configuration q∗q^{*} satisfying

Complete original mathematical display S4.E6
Original selectable mathematical display
Equation 6
F⁡(q∗,q,u):=M​q∗−qδ​t2+∂E⁡(q∗)∂q∗−τ⁡(u)+∑i=1nhJi​(q)⊤​Di​(Ji​(q)​(q∗−q)+ci​(q))=0,F(q^{*};q,u):=M\frac{q^{*}-q}{\delta t^{2}}+\frac{\partial E(q^{*})}{\partial q^{*}}-\tau(u)+\sum_{i=1}^{n_{h}}J_{i}(q)^{\top}D_{i}\,\bigl(J_{i}(q)\,(q^{*}-q)+c_{i}(q)\bigr)=0, (6)
Original mathematical definitions, assumptions and evaluation protocol

where nhn_{h} denotes the number of holonomic constraints, and Di∈ℝ3×3D_{i}\in\mathbb{R}^{3\times 3} is a diagonal stiffness matrix associated with the iith holonomic constraint. When the constraint is active, DiD_{i} is chosen sufficiently large to enforce (4); otherwise, Di=0D_{i}=0. (6) is solved using Newton’s method and implicitly defines a mapping from the current configuration qq to the intermediate configuration q∗q^{*}. In the second stage, we solve the optimization problem

Complete original mathematical display S4.E7
Original selectable mathematical display
Equation 7
minq+\displaystyle\min_{q^{+}} 12​(q+−q∗)⊤​[M+∂2E⁡(q∗)∂(q∗)2+∑i=1nhJi​(q)⊤​Di​Ji​(q)⏟P]​(q+−q∗)\displaystyle\frac{1}{2}(q^{+}-q^{*})^{\top}\,\bigl[\underbrace{M+\frac{\partial^{2}E(q^{*})}{\partial(q^{*})^{2}}+\sum_{i=1}^{n_{h}}J_{i}(q)^{\top}D_{i}J_{i}(q)}_{P}\bigr]\,(q^{+}-q^{*}) (7)
s.t.\displaystyle\text{s.t.} Ji(q∗)(q+−q∗)+[ϕi​(q∗)00]∈ℱμ∗,∀i=nh+1,…,nc.\displaystyle J_{i}(q^{*})\,(q^{+}-q^{*})+\begin{bmatrix}\phi_{i}(q^{*})&0&0\end{bmatrix}\in\mathcal{F}_{\mu}^{*},\quad\forall i=n_{h}\!+\!1,\dots,n_{c}.
Original mathematical definitions, assumptions and evaluation protocol

The KKT conditions of (7) recover the contact constraint in (5) together with P⁡(q+−q∗)=∑i=nh+1ncJi​(q)⊤​λi\textstyle P\,(q^{+}\!-\!q^{*})\!=\!\sum_{i=n_{h}\!+\!1}^{n_{c}}J_{i}(q)^{\top}\,\lambda_{i}, which corresponds to a linearization of the dynamics about the intermediate configuration q∗q^{*}. Given a (q,u)(q,u) pair, solving (6)–(7) gives the next configuration q+q^{+}. The solvers for (6)–(7) are implemented in JAX, allowing GPU-accelerated matrix factorizations.

Original mathematical definitions, assumptions and evaluation protocol

Differentiating Through Simulation To obtain gradients through the dynamics, we differentiate through the optimization problems in (6) and (7). Rather than differentiating through the unrolled iterative solvers, we employ implicit differentiation. For the first-stage problem (6), let θ:=(q,u)\theta:=(q,u) denote the inputs to the dynamics. Differentiating the implicit equation defining q∗q^{*} yields

Complete original mathematical display S4.E8
Original selectable mathematical display
Equation 8
∂F∂q∗​∂q∗∂θ+∂F∂θ=0⟹∂q∗∂θ=−(∂F∂q∗)−1​∂F∂θ.\frac{\partial F}{\partial q^{*}}\frac{\partial q^{*}}{\partial\theta}+\frac{\partial F}{\partial\theta}=0\implies\frac{\partial q^{*}}{\partial\theta}=-\Bigl(\frac{\partial F}{\partial q^{*}}\Bigr)^{-1}\frac{\partial F}{\partial\theta}. (8)
Original mathematical definitions, assumptions and evaluation protocol

For the second-stage problem (7), we differentiate its KKT conditions, denoted by G⁡(q+,λ,q∗)=0G(q^{+},\lambda;q^{*})=0, with respect to q∗q^{*}. This gives

Complete original mathematical display S4.E9
Original selectable mathematical display
Equation 9
K​[∂q+∂q∗∂λ∂q∗]+∂G∂q∗=0⟹∂q+∂q∗=−[I0]​K−1​∂G∂q∗,K\begin{bmatrix}\frac{\partial q^{+}}{\partial q^{*}}\\ \frac{\partial\lambda}{\partial q^{*}}\end{bmatrix}+\frac{\partial G}{\partial q^{*}}=0\implies\frac{\partial q^{+}}{\partial q^{*}}=-\begin{bmatrix}I&0\end{bmatrix}K^{-1}\frac{\partial G}{\partial q^{*}}, (9)
Original mathematical definitions, assumptions and evaluation protocol

where KK denotes the KKT matrix of the optimization problem. The simulator can thus be treated as a differentiable layer. For a scalar objective ℒ\mathcal{L}, the adjoint with respect to θ=(q,u)\theta=(q,u) is given by

Complete original mathematical display S4.E10
Original selectable mathematical display
Equation 10
∂ℒ∂θ=∂ℒ∂q+​∂q+∂q∗​∂q∗∂θ=[∂ℒ∂q+0]​K−1​∂G∂q∗​(∂F∂q∗)−1​∂F∂θ.\frac{\partial\mathcal{L}}{\partial\theta}=\frac{\partial\mathcal{L}}{\partial q^{+}}\frac{\partial q^{+}}{\partial q^{*}}\frac{\partial q^{*}}{\partial\theta}=\begin{bmatrix}\dfrac{\partial\mathcal{L}}{\partial q^{+}}&0\end{bmatrix}K^{-1}\frac{\partial G}{\partial q^{*}}\left(\frac{\partial F}{\partial q^{*}}\right)^{-1}\frac{\partial F}{\partial\theta}. (10)
Original mathematical definitions, assumptions and evaluation protocol

This expression naturally defines a custom backward pass for the simulator layer. Starting from the adjoint of q+q^{+}, we first solve a linear system involving K⊤K^{\top} and compute a vector-Jacobian product with ∂G/∂q∗\partial G/\partial q^{*} to obtain the adjoint of q∗q^{*}. We then solve a second linear system involving (∂F/∂q∗)⊤(\partial F/\partial q^{*})^{\top} and compute a vector-Jacobian product with ∂F/∂θ\partial F/\partial\theta to obtain the adjoint of θ\theta. Since the factorization of KK and ∂F/∂q∗\partial F/\partial q^{*} are byproducts of the forward solve, the backward pass incurs no additional cost.

Original mathematical definitions, assumptions and evaluation protocol

To address this issue, we adopt the idea of surrogate gradients [54], in which the gradient used for planning is replaced by a surrogate estimator that does not correspond to the true gradient obtained by differentiating through the physical rollout. This introduces a controlled bias in exchange for improved gradient information in regimes where the true gradient is uninformative. The holonomic constraint in (4) is active only when ci​(q)=0c_{i}(q)=0, at which point a nonzero stiffness matrix DiD_{i} is used in (6) to enforce the constraint. This switching between inactive and active constraints introduces both discontinuities and vanishing gradients. To smooth this transition, we redefine DiD_{i} as Di=Di,maxexp(−∥ci(q)∥2/κ)D_{i}=D_{i,\max}\exp(-\|c_{i}(q)\|_{2}/\kappa), where κ\kappa is a smoothing parameter. As κ→0\kappa\to 0, this formulation approaches the original indicator-like activation behavior.

Original mathematical definitions, assumptions and evaluation protocol

The contact constraint in (5), which encodes non-contact, stiction, and sliding regimes, similarly introduces discontinuous and vanishing gradients. We smooth this by relaxing the complementarity condition in (5) to (Ji​(q)​(q+−q)+[ϕi​(q)00])⊤​λi=κ(J_{i}(q)\,(q^{+}-q)+\begin{bmatrix}\phi_{i}(q)&0&0\end{bmatrix})^{\top}\lambda_{i}=\kappa, which recovers the original complementarity constraint when κ=0\kappa=0. The smoothed dynamics is used solely for gradient computation and the nonsmooth dynamics are used for forward rollouts, preserving model accuracy.

4.2 Simulator-Informed GPU-Parallel Output Feedback SLS

Equations 11–12 formulate finite-horizon robust output feedback and its SLS approximation, using nominal linearizations and row-wise norm bounds to tighten constraints. Equations 13–17 preserve the complete Kalman/observer recursions and associative combination rules, including all five rows of Equation 12. GPU scans reduce serial horizon dependence, while dense state-space operations and memory remain. The text describes affine constraints and refers nonlinear treatment to prior work. Any claimed tube certificate requires valid uncertainty/remainder treatment and successful synthesis; merely plotting a tube or computing a surrogate Jacobian does not establish those prerequisites for arbitrary nonlinear contact dynamics.

Original mathematical definitions, assumptions and evaluation protocol

To plan robust output feedback policies for the deformable object, we wish to approximately solve the idealized optimization (11) to obtain the output-feedback policy 𝝅:=(π0​(⋅),…​πN−1​(⋅))\boldsymbol{\pi}:=(\pi_{0}(\cdot),\ldots\pi_{N-1}(\cdot)),

Complete original mathematical display S4.E11
Original selectable mathematical display
Equation 11
min𝝅\displaystyle\min_{\boldsymbol{\pi}} J⁡(𝝅)\displaystyle J(\boldsymbol{\pi}) (11)
s.t.\displaystyle\text{s.t.} qk+1=f∗​(qk,uk)+E⁡(qk)​wk,\displaystyle q_{k+1}=f^{*}(q_{k},u_{k})+E(q_{k})w_{k}, ∀k∈[N],q0∈𝒬0,\displaystyle\forall k\in[N],\qquad q_{0}\in\mathcal{Q}_{0},
yk+1=h⁡(qk)+F⁡(qk)​ek,\displaystyle y_{k+1}=h(q_{k})+F(q_{k})e_{k}, ∀k∈[N],uk=πk(y0,…,yk),\displaystyle\forall k\in[N],\qquad u_{k}=\pi_{k}(y_{0},\ldots,y_{k}), ∀k∈[N],\displaystyle\forall k\in[N],\hskip-5.0pt
(qk,uk)∈𝒮,\displaystyle(q_{k},u_{k})\in\mathcal{S}, ∀wk∈ℬnq,∀ek∈ℬny,∀k∈[N].\displaystyle\forall w_{k}\in\mathcal{B}^{n_{q}},\;\forall e_{k}\in\mathcal{B}^{n_{y}},\;\forall k\in[N].
Original mathematical definitions, assumptions and evaluation protocol

where [N]:={0,…,N−1}[N]:=\{0,\ldots,N-1\}, 𝒬0\mathcal{Q}_{0} denotes the set of possible initial conditions, 𝒮={(q,u)∈ℝnq+nu|ci⊤(q,u)+bi≤0,i=1,…,nc}\mathcal{S}=\left\{(q,u)\in\mathbb{R}^{n_{q}+n_{u}}\;\middle|\;c_{i}^{\top}(q,u)+b_{i}\leq 0,\;i=1,\ldots,n_{c}\right\} defines the safe set. While we consider nonlinear constraints in Sec. 5, to simplify notation, we describe the method for the case of affine constraints. Nonlinear constraints can be robustly enforced following the approach of [55]. The dynamics and observations are as defined in (1). To efficiently solve (11), we extend GPU-SLS [49], which is limited to state-feedback (i.e., requires perfect state knowledge), to the output-feedback setting by building on [52]. Following [52], the output-feedback SLS synthesis problem can be written as,

Complete original mathematical display S4.E12
Original selectable mathematical display
Equations 12a–12e
min𝐳,𝐯,𝚽,𝝉\displaystyle\hskip-10.0pt\min_{\mathbf{z},\mathbf{v},\mathbf{\Phi},\boldsymbol{\tau}}\quad J⁡(𝐳,𝐯,𝝉,𝚽)\displaystyle J(\mathbf{z},\mathbf{v},\boldsymbol{\tau},\mathbf{\Phi}) (12a)
s.t. zk+1=f⁡(zk,vk),∀k∈[N]\displaystyle z_{k+1}=f(z_{k},v_{k}),\qquad\forall k\in[N] (12b)
[𝐈−𝐙𝐀⁡(𝐳,𝐯),−𝐙𝐁⁡(𝐳,𝐯)]​𝚽=[𝐈, 0],𝚽​[(𝐈−𝐙𝐀⁡(𝐳,𝐯))⊤,(−𝐙𝐂⁡(𝐳,𝐯))⊤]⊤=[𝐈, 0]⊤,\displaystyle\hskip-4.0pt\begin{bmatrix}\mathbf{I-ZA(z,v)},\ \mathbf{-ZB(z,v)}\end{bmatrix}\mathbf{\Phi}=\begin{bmatrix}\mathbf{I},\ \mathbf{0}\end{bmatrix},\ \mathbf{\Phi}\begin{bmatrix}(\mathbf{I-ZA(z,v)})^{\top}\hskip-6.0pt,\ \mathbf{(-ZC(z,v))^{\top}}\end{bmatrix}^{\top}=\begin{bmatrix}\mathbf{I},\ \mathbf{0}\end{bmatrix}^{\top}\hskip-8.0pt, (12c)
∑j=0k−1‖ci⊤​𝚽k,jw​Ej‖2,r+‖ci⊤​𝚽k,je​Fj‖2,r+ci⊤​[zkvk]≤−bi,∀i∈[nc],∀k∈[N]\displaystyle\textstyle\sum_{j=0}^{k-1}\left\|c_{i}^{\top}\mathbf{\Phi}_{k,j}^{\text{w}}E_{j}\right\|_{2,r}\hskip-3.0pt+\left\|c_{i}^{\top}\mathbf{\Phi}_{k,j}^{\text{e}}F_{j}\right\|_{2,r}\hskip-3.0pt+c_{i}^{\top}\begin{bmatrix}z_{k}\\ v_{k}\end{bmatrix}\leq-b_{i},\qquad\forall i\in[n_{c}],\;\forall k\in[N] (12d)
∑j=0k−1‖𝚽k,jw​Ej‖2,r+‖𝚽k,je​Fj‖2,r≤τk,∀k∈[N]\displaystyle\textstyle\sum_{j=0}^{k-1}\left\|\mathbf{\Phi}_{k,j}^{\text{w}}E_{j}\right\|_{2,r}+\left\|\mathbf{\Phi}_{k,j}^{\text{e}}F_{j}\right\|_{2,r}\leq\tau_{k},\qquad\forall k\in[N] (12e)
Original mathematical definitions, assumptions and evaluation protocol

where 𝐀⁡(𝐳,𝐯),𝐁⁡(𝐳,𝐯)\mathbf{A(z,v),B(z,v)} and 𝐂⁡(𝐳,𝐯)\mathbf{C(z,v)} denote the linearizations of the dynamics and observation models about the nominal trajectory (𝐳,𝐯)\mathbf{(z,v)}, ∥⋅∥2,r\|\cdot\|_{2,r} denotes the row-wise two norm, and EjE_{j} and FjF_{j} are the corresponding linearized state and observation disturbance scalings. The matrix 𝐙\mathbf{Z} denotes the block downshift operator, with InqI_{n_{q}} on the first subdiagonal and zeros elsewhere. The closed-loop response operators 𝚽w\mathbf{\Phi}^{\text{w}} and 𝚽e\mathbf{\Phi}^{\text{e}} characterize the propagation of process and observation disturbances, respectively, via constraint (12c), and τk\tau_{k} defines the reachable tube width. (12d)–(12e) compute output-feedback reachable tubes under disturbances wkw_{k} and eke_{k}, and use them to tighten the constraints so the closed-loop reachable set remains inside 𝒮\mathcal{S}. To solve (12), [52] decomposes the optimization into three stages: 1) a nominal trajectory generation (𝐳,𝐯)\mathbf{(z,v)}, 2) state feedback controller optimization 𝚽¯\mathbf{\bar{\Phi}}, and lastly 3) an output feedback controller optimization 𝚽^\mathbf{\hat{\Phi}}. The first two stages are handled directly by GPU-SLS [49]. We therefore focus on the parallel implementation of the third stage by expressing the observer recursions as associative operators, enabling GPU-parallel prefix scans. An optimal solution of the output controller can be found by one backward Kalman recursion,

Complete original mathematical display S4.E13
Original selectable mathematical display
Equation 13
Πk,0=Ξ​Ξ⊤;Lk,j+1\displaystyle\Pi_{k,0}=\Xi\Xi^{\top};\quad L_{k,j+1} =−(Fj​Fj⊤+Cj​Πk,j​Cj⊤)−1​Cj​Πk,j​Aj⊤\displaystyle=-(F_{j}F_{j}^{\top}+C_{j}\Pi_{k,j}C_{j}^{\top})^{-1}C_{j}\Pi_{k,j}A_{j}^{\top} (13)
Πk,j+1\displaystyle\Pi_{k,j+1} =Ej​Ej⊤+Aj​Πk,j​Aj⊤+Aj​Πk,j​Cj⊤​Lk,j+1,\displaystyle=E_{j}E_{j}^{\top}+A_{j}\Pi_{k,j}A_{j}^{\top}+A_{j}\Pi_{k,j}C^{\top}_{j}L_{k,j+1},
Original mathematical definitions, assumptions and evaluation protocol

where Ξ\Xi denotes the initial state uncertainty, and NN independent forward propagations,

Complete original mathematical display S4.E14
Original selectable mathematical display
Equation 14
𝚽^k,kx=I;\displaystyle\hat{\mathbf{\Phi}}^{x}_{k,k}=I; 𝚽^k,jy=𝚽^k,jx​Lk,j⊤,\displaystyle\hat{\mathbf{\Phi}}^{y}_{k,j}=\hat{\mathbf{\Phi}}^{x}_{k,j}L^{\top}_{k,j}, 𝚽^xk,j=𝚽^xk,j+1(Aj+Lk,j⊤Cj),∀k∈[N],∀j∈[k],\displaystyle\hat{\mathbf{\Phi}}^{x}_{k,j}=\hat{\mathbf{\Phi}}^{x}_{k,j+1}(A_{j}+L_{k,j}^{\top}C_{j}),\quad\forall k\in[N],\ \ \forall j\in[k], (14)
Original mathematical definitions, assumptions and evaluation protocol

The matrices 𝚽w\mathbf{\Phi}^{\text{w}} and 𝚽e\mathbf{\Phi}^{\text{e}} can then be assembled by using 𝐌=𝐈−𝐙𝐀\mathbf{M=I-ZA} and

Complete original mathematical display S4.E15
Original selectable mathematical display
Equation 15
𝚽xw=𝚽¯x+𝚽^x−𝚽¯x​𝐌​𝚽^x,\displaystyle\mathbf{\Phi}^{\text{xw}}=\bar{\mathbf{\Phi}}^{\text{x}}+\hat{\mathbf{\Phi}}^{\text{x}}-\bar{\mathbf{\Phi}}^{\text{x}}\mathbf{M}\hat{\mathbf{\Phi}}^{\text{x}}, 𝚽uw=𝚽¯u​(𝐈−𝐌​𝚽^𝐱),\displaystyle\mathbf{\Phi}^{\text{uw}}=\bar{\mathbf{\Phi}}^{\text{u}}\mathbf{(I-M\hat{\Phi}^{x})}, 𝚽xe=𝚽^y−𝚽¯x​𝐌​𝚽^y,\displaystyle\mathbf{\Phi}^{\text{xe}}=\mathbf{\hat{\Phi}}^{\text{y}}-\bar{\mathbf{\Phi}}^{\text{x}}\mathbf{M}\hat{\mathbf{\Phi}}^{\text{y}}, 𝚽ue=−𝚽¯u​𝐌​𝚽^y.\displaystyle\mathbf{\Phi}^{\text{ue}}=-\bar{\mathbf{\Phi}}^{\text{u}}\mathbf{M}\hat{\mathbf{\Phi}}^{\text{y}}. (15)
Original mathematical definitions, assumptions and evaluation protocol

Both (13) and (14) require dense matrix multiplications serially through the planning horizon. As such, we use parallel associative scans on the GPU. We define the conditional value function, Vi→l​(𝚽k,i,𝚽k,l)=maxγ⁡12​𝚽k,i⊤​P~i,l​𝚽k,i−12​γ⊤​D~i,l​γ−γ⊤​(𝚽k,l−A~i,l​𝚽k,i)V_{i\rightarrow l}(\mathbf{\Phi}_{k,i},\mathbf{\Phi}_{k,l})\!=\!\max_{\gamma}\tfrac{1}{2}\mathbf{\Phi}_{k,i}^{\top}\tilde{P}_{i,l}\mathbf{\Phi}_{{k,i}}-\tfrac{1}{2}\gamma^{\top}\tilde{D}_{i,l}\gamma-\gamma^{\top}(\mathbf{\Phi}_{k,l}-\tilde{A}_{i,l}\mathbf{\Phi}_{k,i}), and combination rules,

Complete original mathematical display S4.E16
Original selectable mathematical display
Equation 16
P~i,l\displaystyle\tilde{P}_{i,l} =P~i,m⊗P~m,l:=(A~i,m⊤(I+P~m,lD~i,m)−1P~m,lA~i,m+P~i,m,\displaystyle=\tilde{P}_{i,m}\otimes\tilde{P}_{m,l}:=(\tilde{A}_{i,m}^{\top}(I+\tilde{P}_{m,l}\tilde{D}_{i,m})^{-1}\tilde{P}_{m,l}\tilde{A}_{i,m}+\tilde{P}_{i,m}, (16)
A~i,l\displaystyle\tilde{A}_{i,l} =A~i,m⊗A~m,l:=A~m,l​(I+D~i,m​P~m,l)−1​A~i,m,\displaystyle=\tilde{A}_{i,m}\otimes\tilde{A}_{m,l}:=\tilde{A}_{m,l}(I+\tilde{D}_{i,m}\tilde{P}_{m,l})^{-1}\tilde{A}_{i,m},
D~i,l\displaystyle\tilde{D}_{i,l} =D~i,m⊗D~m,l:=A~m,l​(I+D~i,m​P~m,l)−1​D~i,m​(A~m,l)⊤+D~m,l,\displaystyle=\tilde{D}_{i,m}\otimes\tilde{D}_{m,l}:=\tilde{A}_{m,l}(I+\tilde{D}_{i,m}\tilde{P}_{m,l})^{-1}\tilde{D}_{i,m}(\tilde{A}_{m,l})^{\top}+\tilde{D}_{m,l},
Complete original mathematical display S4.E17
Original selectable mathematical display
Equation 17
A~i+1,i=Ai⊤,D~i+1,i=Ci⊤(FiFi⊤)−1Ci,P~i+1,i=EiEi⊤,P~1,0=ΞΞ⊤,A~1,0=D~1,0=0.\begin{gathered}\tilde{A}_{i+1,i}=A_{i}^{\top},\ \ \ \tilde{D}_{i+1,i}=C_{i}^{\top}(F_{i}F_{i}^{\top})^{-1}C_{i},\ \ \ \tilde{P}_{i+1,i}=E_{i}E_{i}^{\top},\ \ \ \tilde{P}_{1,0}=\Xi\Xi^{\top},\ \ \ \tilde{A}_{1,0}=\tilde{D}_{1,0}=0.\end{gathered} (17)
Original mathematical definitions, assumptions and evaluation protocol

We can then recover Kalman gains Lk,jL_{k,j} through a forward parallel associative scan using (16)–(17) to recover Πk,j=P~0,j:=P~0,1⊗P~1,2⊗⋯⊗P~j−1,j\Pi_{k,j}=\tilde{P}_{0,j}:=\tilde{P}_{0,1}\otimes\tilde{P}_{1,2}\otimes\cdots\otimes\tilde{P}_{j-1,j}, and then Lk,jL_{k,j} from (13). To retrieve 𝚽^\mathbf{\hat{\Phi}}, we use a similar procedure and define the conditional observer propagation N^i→l(k)​(𝚽^k,ix,𝚽^k,lx)=𝚽^k,ix​A^i,l(k)\hat{N}^{(k)}_{i\rightarrow l}(\hat{\mathbf{\Phi}}^{x}_{k,i},\hat{\mathbf{\Phi}}^{x}_{k,l})=\hat{\mathbf{\Phi}}^{x}_{k,i}\hat{A}_{i,l}^{(k)} for each disturbance injection kk with combination rule A^m,l(k)​A^i,m(k)\hat{A}^{(k)}_{m,l}\hat{A}^{(k)}_{i,m} and initialization A^i+1,i(k)=Ai+Lk,i⊤​Ci\hat{A}^{(k)}_{i+1,i}=A_{i}+L_{k,i}^{\top}C_{i}. We obtain the solution to (14) by a forward parallel associative scan, 𝚽^xk,i=N^j+1→j(k)⊗N^j→j−1(k)⊗⋯⊗N^i+1→i(k)\hat{\mathbf{\Phi}}^{x}_{k,i}=\hat{N}_{j+1\rightarrow j}^{(k)}\otimes\hat{N}_{j\rightarrow j-1}^{(k)}\otimes\cdots\otimes\hat{N}_{i+1\rightarrow i}^{(k)}, and 𝚽^k,jy=𝚽^k,jx​Lk,j⊤\hat{\mathbf{\Phi}}^{y}_{k,j}=\hat{\mathbf{\Phi}}^{x}_{k,j}L^{\top}_{k,j}, solving the observer-dependent closed-loop response in logarithmic time, analogous to the complexity of GPU-SLS.

4.3 Accelerating Model-Based RL via Analytical Policy Gradients (APG)

The simulator supports expert-data generation and an analytical policy-gradient learning variant, APG. These learned policies are evaluated separately from the reachability-constrained SLS controller and do not inherit its constraint guarantees. The PPO lift baseline is warm-started with ten CORD-SLS demonstrations; without that assistance it fails to discover contact within the stated interaction budget. Training curves address cost/sample efficiency, whereas main-table endpoints use matched total interaction budgets.

4.4 Calibrated Reachability Guarantees via Conformal Prediction (CP)

Calibration measures the aggregate next perceived-state residual, combining dynamics and perception effects. It does not independently identify latent process noise. Under closed-loop/calibration-score exchangeability, the formal method allocates trajectory miscoverage across steps and selects the augmented finite-sample quantile, returning infinity when too few samples support the requested level. The theorem then assumes valid perceived-state/control containment and inflates state tubes by a deterministic perception-error boundρ. This bound must cover reconstructed-state error, including tracking and occlusion effects, rather than sensor depth noise alone. With 100 samples, a finite formal radius requires per-step miscoverage at least 1/101; for an illustrative 95% guarantee over 10 steps, the stated index is 101 and hence infinity. This arithmetic limits what the calibration can certify; it is not evidence of a robot failure. Random-control calibration alone does not demonstrate the assumed closed-loop exchangeability.

Original mathematical definitions, assumptions and evaluation protocol

The SLS formulation in Sec. 4.2 requires uncertainty sets for the closed-loop dynamics and observation model to certify safety for the real system (1). On hardware, however, independent labels for the true process disturbance or state-estimation error are typically unavailable. Instead, we measure the discrepancy between the model-predicted next observation and the next observation returned by perception, and use CP to calibrate an aggregate one-step prediction error in the controller coordinates. In our implementation (Sec. 5), the perception estimates the full rope state from RGB-D images. Thus, in (1b), we take h⁡(q)=qh(q)=q, so yk+1=qk+eky_{k+1}=q_{k}+e_{k} with ‖ek‖2≤ρ\|e_{k}\|_{2}\leq\rho, where ρ\rho is a worst-case measurement-error bound from camera accuracy specifications. We define the perceived state q¯k:=yk+1\bar{q}_{k}:=y_{k+1} as the noisy full-state estimate used by the controller. Although the physical system evolves on latent qkq_{k}, our SLS implementation plans over q¯k\bar{q}_{k}; the CP-calibrated residual certifies tubes for q¯k\bar{q}_{k}, which are conservatively inflated by ρ\rho to obtain true-state tubes.

Original mathematical definitions, assumptions and evaluation protocol

Let 𝒟cal:={(q¯i,ui,q¯i+)}i=1n\mathcal{D}_{\mathrm{cal}}:=\{(\bar{q}_{i},u_{i},\bar{q}_{i}^{+})\}_{i=1}^{n} be a held-out calibration set of perceived-state transitions, where q¯i+\bar{q}_{i}^{+} is the next perceived state; equivalently, a sample is (yk+1,uk,yk+2)(y_{k+1},u_{k},y_{k+2}). We define nonconformity scores si:=‖q¯i+−f⁡(q¯i,ui)‖2s_{i}:=\|\bar{q}_{i}^{+}-f(\bar{q}_{i},u_{i})\|_{2}, and let s(1)≤⋯≤s(n)s_{(1)}\leq\cdots\leq s_{(n)} be the sorted scores. For trajectory-level miscoverage δ\delta over horizon TT, set δ¯:=δ/T\bar{\delta}:=\delta/T and q:=s(⌈(n+1)​(1−δ¯)⌉)q:=s_{(\lceil(n+1)(1-\bar{\delta})\rceil)}, with q=+∞q=+\infty if the index exceeds nn. The CP set for the next perceived state is 𝒞⁡(q¯,u):={q¯+∈𝒬|‖q¯+−f⁡(q¯,u)‖2≤rCP}\mathcal{C}(\bar{q},u):=\left\{\bar{q}^{+}\in\mathcal{Q}\;\middle|\;\|\bar{q}^{+}-f(\bar{q},u)\|_{2}\leq r_{\textrm{CP}}\right\}. We instantiate SLS on the perceived-state surrogate q¯k+1=f⁡(q¯k,uk)+rCP​I​wk\bar{q}_{k+1}=f(\bar{q}_{k},u_{k})+r_{\textrm{CP}}Iw_{k}, with wk∈ℬnqw_{k}\in\mathcal{B}^{n_{q}}. This is a calibrated model of the full-state estimate used by the controller, rather than an independently identified process-noise model for the latent state qkq_{k}. Let 𝒯¯kq\bar{\mathcal{T}}_{k}^{q} and 𝒯ku\mathcal{T}_{k}^{u} denote the resulting perceived-state and control reachable tubes. Assuming exchangeability between closed-loop residual scores ‖q¯k+1−f⁡(q¯k,uk)‖2\|\bar{q}_{k+1}-f(\bar{q}_{k},u_{k})\|_{2} and the calibration scores, CP and robust SLS ensure (proof in App. A):

Theorem 1 (True-state containment from CP tubes).

Let yk+1=qk+eky_{k+1}=q_{k}+e_{k} with ‖ek‖2≤ρ\|e_{k}\|_{2}\leq\rho, and define q¯k:=yk+1\bar{q}_{k}:=y_{k+1}. Suppose conformal calibration and robust SLS synthesis yield perceived-state and control tubes 𝒯¯kq\bar{\mathcal{T}}_{k}^{q} and 𝒯ku\mathcal{T}_{k}^{u} satisfying ℙ⁡[q¯k∈𝒯¯kq,uk∈𝒯ku,k=0,…,N]≥1−δ\mathbb{P}\left[\bar{q}_{k}\in\bar{\mathcal{T}}_{k}^{q},\;u_{k}\in\mathcal{T}_{k}^{u},\;k=0,\ldots,N\right]\geq 1-\delta. Then the true state and control trajectories satisfy ℙ⁡[qk∈𝒯¯kq⊕ρ​𝔹nq,uk∈𝒯ku,k=0,…,N]≥1−δ\mathbb{P}\left[q_{k}\in\bar{\mathcal{T}}_{k}^{q}\oplus\rho\mathbb{B}^{n_{q}},\;u_{k}\in\mathcal{T}_{k}^{u},\;k=0,\ldots,N\right]\geq 1-\delta.

Original mathematical definitions, assumptions and evaluation protocol

Accordingly, we define the true-state tube as 𝒯kq:=𝒯¯kq⊕ρ​ℬnq\mathcal{T}_{k}^{q}:=\bar{\mathcal{T}}_{k}^{q}\oplus\rho\mathcal{B}^{n_{q}}. Thus, to enforce true-state safety constraints qk∈𝒬safeq_{k}\in\mathcal{Q}_{\mathrm{safe}}, it suffices to impose the tightened perceived-state condition 𝒯¯kq⊆𝒬safe⊖ρ​ℬnq\bar{\mathcal{T}}_{k}^{q}\subseteq\mathcal{Q}_{\mathrm{safe}}\ominus\rho\mathcal{B}^{n_{q}}. State-dependent uncertainty bounds can be incorporated by replacing the scalar radius rCPr_{\textrm{CP}} with a calibrated state-dependent bound, e.g., by adapting [56] to the perceived-state setting.

5 Results

The experiments evaluate contact discovery, simulated safety/goal error, learning curves, output-feedback rollouts and hardware feasibility. Dynamics-evaluation counts, online planning time, offline training interactions and policy inference time have different meanings. Safety percentages in a benchmark do not substitute for verifying the formal assumptions. The complete tables retain results that favor other methods on selected accuracy, speed or goal metrics.

Complete original Table 1: Runtime and performance metrics across tasks.
Table 1: Runtime and performance metrics across tasks.
Accessible original table, complete values and grouped headers
Task Method
Dynamics
Evaluations Count
Execution Time
Per-Step [s]
Safety Rate [%] Goal Error [m]
Lift Rope Ours 5400 0.050 ±\pm 0.001 100.0 ±\pm 0.0 0.0548 ±\pm 0.0003
MPPI 1024000 0.177 ±\pm 0.043 100.0 ±\pm 0.0 0.0681 ±\pm 0.0038
PPO 16000000 0.0004 ±\pm 0.0001 96.5 ±\pm 0.2 0.0587 ±\pm 0.0016
APG 16000000 0.0004 ±\pm 0.0001 100.0 ±\pm 0.0 0.0573 ±\pm 0.0017
Drag Rope Ours 13500 0.070 ±\pm 0.012 99.9 ±\pm 0.1 0.0360 ±\pm 0.0077
MPPI 2560000 0.234 ±\pm 0.018 1.0 ±\pm 0.1 0.0814 ±\pm 0.0034
PPO 40000000 0.0004 ±\pm 0.0001 97.0 ±\pm 0.4 0.0320 ±\pm 0.0092
APG 40000000 0.0004 ±\pm 0.0001 93.0 ±\pm 0.4 0.0302 ±\pm 0.0061
Fold cloth Ours 3400 0.758 ±\pm 0.107 100.0 ±\pm 0.0 0.0611 ±\pm 0.0072
MPPI 64000 18.81 ±\pm 6.11 64.0 ±\pm 6.1 0.0738 ±\pm 0.0079
Flatten cloth Ours 3400 0.711 ±\pm 0.077 100.0 ±\pm 0.0 0.0234 ±\pm 0.0089
MPPI 64000 11.10 ±\pm 4.66 80.0 ±\pm 0.2 0.0526 ±\pm 0.0102

Simulation Benchmark Comparison

AppendixF compares forward/Jacobian runtimes for a 20-node rope and 100-node cloth, then evaluates settled hardware rope prediction using a different 10-link configuration. Maximum speedups are setting-specific: rope forward 1.095 ms versus DaXBench 786.48 ms is about 718×, while Jacobian 5.198 ms versus 11561.57 ms is about 2224×. Cloth speedups are much smaller. DEFORM has lower prediction error than CORD-SLS, so the speed benefit comes with a fidelity tradeoff rather than uniform accuracy superiority.

Original mathematical definitions, assumptions and evaluation protocol

See App. F for baseline comparisons of our differentiable simulator against prior simulators [9, 36] in terms of accuracy and runtime. We show that our simulator achieves comparable accuracy while accelerating forward simulation and Jacobian computation by up to 700×\times and 2000×\times, respectively. This efficiency, along with the integration of contact smoothing, enables real-time control rates and low goal error in our deformable object planning experiments, which we discuss next.

Planning Benchmark Comparison on Deformable Manipulation Tasks

Table 1 gives rope planner times 50/70 ms versus MPPI 177/234 ms and cloth 758/711 ms versus 18.81/11.10 seconds. Learned policies infer in.4 ms, faster than the online planner. CORD-SLS reports 100/99.9/100/100% safety across lift/drag/fold/flatten, but drag PPO/APG achieve lower goal error at 97/93% safety. APG's cost improvement is not improved safety over PPO in that task. Cloth has no RL comparison. Different tasks need their own costs/constraints; the common optimizer's reuse is not an unchanged learned multitask policy. Rollout-count/uncertainty semantics for Table 1 are not fully specified, so no new significance claim is inferred.

Complete original Figure 2: Timelapse of our method on each task. (a) Rope lifting, (b) Rope dragging around an obstacle with uncertainty tubes, (c) Cloth folding, (d) Cloth flattening. By formulating control as trajectory optimization, our method generalizes across diverse tasks without task-specific retraining.
Figure 2: Timelapse of our method on each task. (a) Rope lifting, (b) Rope dragging around an obstacle with uncertainty tubes, (c) Cloth folding, (d) Cloth flattening. By formulating control as trajectory optimization, our method generalizes across diverse tasks without task-specific retraining.
Original selectable panel captions and notation

(a)

(b)

(c)

(d)

Complete original Figure 3: (a) : Solved trajectory for a rope manipulation task using CORD-SLS. (b) : Robust tubes for four rope nodes and their corresponding rollouts. All rollouts remain within their tubes, demonstrating robustness to both dynamical error and measurement uncertainty. See App. E for reachable tubes for all state dimensions.
Figure 3: (a): Solved trajectory for a rope manipulation task using CORD-SLS. (b): Robust tubes for four rope nodes and their corresponding rollouts. All rollouts remain within their tubes, demonstrating robustness to both dynamical error and measurement uncertainty. See App. E for reachable tubes for all state dimensions.

Output Feedback

Figures 3/9 show output-feedback rope rollouts under simulated process and measurement disturbances, with the full 39-state tube grid retained. They illustrate containment for the synthesized finite-horizon observer/controller and sampled disturbances. This evidence differs from executing a sequence of newly replanned controllers or certifying unseen perception failures.

Hardware

Hardware uses two KUKA iiwa arms, RGB-D keypoint reconstruction and separate perception/solver GPUs. The paper collects 100 random-control transitions and chooses a rounded.03 m radius covering 87% of calibration residuals. That is empirical in-sample coverage, not the formal trajectory-level quantile guarantee. Two tasks demonstrate rope obstacle traversal and cloth folding; no repeated-trial success denominator or measured trajectory-level confidence is supplied. Four project movies are two views of these behaviors, not four independent statistical trials. Hardware solve time below 20 ms and slower perception differ from Table 1's benchmark timings.

6 Discussion, Limitations, and Conclusion

The useful contribution is an integrated fast simulator and robust-planning interface with a clear finite-horizon containment argument. Experimental results show feasible contact-rich rope/cloth control and favorable planning costs in selected settings. Conclusions about certified real-world safety remain conditional on calibration, full-state estimation error bounds, nonlinear tube validity and the actual controller used during the horizon.

Limitations

The paper identifies contact-smoothing error, occlusion/topology-sensitive perception and memory growth with state dimension/horizon as limitations. Proposed remedies—smoothing-error tubes, better sensing/learned observations and low-rank responses—are future or referenced extensions. The reported implementation does not thereby acquire these additional protections.

Original hardware recordings and interactive rollouts

All four original WebM hardware recordings are preserved locally with unchanged source bytes and derived poster frames. Side/front camera pairs depict rope and cloth behavior. Original interactive Viser simulations and the overview video remain linked at their source hosts; the edition does not claim local copies of those viewers. Interactive rollouts are simulation illustrations, while multiple camera views do not expand the hardware trial count.

Leveraging GPU-accelerated contact-smoothed differentiable simulation and output-feedback system level synthesis, we enable real-time manipulation of deformable objects involving intermittent contact while remaining robust to dynamics and visual uncertainty.
Leveraging GPU-accelerated contact-smoothed differentiable simulation and output-feedback system level synthesis, we enable real-time manipulation of deformable objects involving intermittent contact while remaining robust to dynamics and visual uncertainty.
Leveraging GPU-accelerated contact-smoothed differentiable simulation and output-feedback system level synthesis, we enable real-time manipulation of deformable objects involving intermittent contact while remaining robust to dynamics and visual uncertainty.
Leveraging GPU-accelerated contact-smoothed differentiable simulation and output-feedback system level synthesis, we enable real-time manipulation of deformable objects involving intermittent contact while remaining robust to dynamics and visual uncertainty.

Appendix A Proofs

Both the main theorem and its appendix restatement are preserved in full, together with the proof. The proof uses q=perceived-state minus measurement error: on a valid perceived-state containment event, adding the error ball contains the true state. It carries the assumed event's probability lower bound forward. It does not independently prove the antecedent SLS containment, exchangeability, a numeric hardwareρ bound or recursive feasibility. The two original proof displays remain alongside their selectable source.

Theorem A.1 (True-state containment from perceived-state conformal tubes).

Let yk+1=qk+eky_{k+1}=q_{k}+e_{k} with ‖ek‖2≤ρ\|e_{k}\|_{2}\leq\rho, and define q¯k:=yk+1\bar{q}_{k}:=y_{k+1}. Suppose conformal calibration and robust SLS synthesis yield perceived-state and control tubes 𝒯¯kq\bar{\mathcal{T}}_{k}^{q} and 𝒯ku\mathcal{T}_{k}^{u} satisfying ℙ⁡[q¯k∈𝒯¯kq,uk∈𝒯ku,k=0,…,N]≥1−δ\mathbb{P}\left[\bar{q}_{k}\in\bar{\mathcal{T}}_{k}^{q},\;u_{k}\in\mathcal{T}_{k}^{u},\;k=0,\ldots,N\right]\geq 1-\delta. Then the true state and control trajectories satisfy ℙ⁡[qk∈𝒯¯kq⊕ρ​𝔹nq,uk∈𝒯ku,k=0,…,N]≥1−δ\mathbb{P}\left[q_{k}\in\bar{\mathcal{T}}_{k}^{q}\oplus\rho\mathbb{B}^{n_{q}},\;u_{k}\in\mathcal{T}_{k}^{u},\;k=0,\ldots,N\right]\geq 1-\delta.

Proof.

On the event that q¯k∈𝒯¯kq\bar{q}_{k}\in\bar{\mathcal{T}}_{k}^{q} and uk∈𝒯kuu_{k}\in\mathcal{T}_{k}^{u} for all kk, we have

Complete original mathematical display A1.Ex1
Original selectable mathematical display
Equation
qk=q¯k−ek.q_{k}=\bar{q}_{k}-e_{k}.

Since ‖ek‖2≤ρ\|e_{k}\|_{2}\leq\rho,

Complete original mathematical display A1.Ex2
Original selectable mathematical display
Equation
qk∈𝒯¯kq⊕ρ​𝔹nqq_{k}\in\bar{\mathcal{T}}_{k}^{q}\oplus\rho\mathbb{B}^{n_{q}}

for all kk. The perceived-state containment event has probability at least 1−δ1-\delta, so the inflated true-state containment event has the same probability lower bound. ∎

Appendix B Perception

Two Orbbec Gemini 336 L cameras are calibrated to the robot workstation. An RTX4090 handles perception/robot communication and an RTX4070 Ti Super runs the solver, connected through ROS2/Ethernet. The visual state is reconstructed through segmentation, tracking, depth and structural assumptions, so camera specifications alone need not bound the entire reconstructed state. Both perception diagrams retain every panel.

B.1 Rope

The braided rope is about 1 cm thick and 1.5 m long, tied to an effective.8 m span and modeled with 10 links on hardware. SAM3 with the rope prompt initializes/re-grounds a skeleton tracker; TAPNext++ tracks between updates, then depth/splines reconstruct ordered 3 D nodes. Re-grounding every 1/5 seconds gives roughly 15/25 Hz tracking, slower than the under 20 ms solver. A faster post-grasp reconstruction instead assumes gravitational sag from gripper poses and known length. Its speed benefit depends on that shape assumption and is distinct from continuously observing every node.

Complete original Figure 4: Perception pipeline for rope state estimation. Raw front-camera images (a) are processed by a SAM3 segmentation network (b). The resulting mask is skeletonized (c), and skeleton points are tracked using TAPNext++ (d). RGB points are then associated with their corresponding RGB-D depth map (e). A spline is fit through the recovered 3D points (f), and fixed inter-node distances are sampled along the spline (g) to reconstruct the rope state (h).
Figure 4: Perception pipeline for rope state estimation. Raw front-camera images (a) are processed by a SAM3 segmentation network (b). The resulting mask is skeletonized (c), and skeleton points are tracked using TAPNext++ (d). RGB points are then associated with their corresponding RGB-D depth map (e). A spline is fit through the recovered 3D points (f), and fixed inter-node distances are sampled along the spline (g) to reconstruct the rope state (h).

B.2 Cloth

The 38×38 cm cloth uses an 8×8 node lattice plus grippers for 198 state coordinates, unlike the 300-DoF simulation benchmark. Perception assumes each row is straight across cloth width, reconstructs a centerline/boundary spline and fills occluded geometry using known length. The optimization constrains gripper x/z differences within 5 cm to support this simplification. This task-specific prior makes folding tractable but limits claims about arbitrary wrinkles, topology or general occluded cloth estimation.

Complete original Figure 5: Perception pipeline for cloth state estimate. Raw front-camera images (a) are processed by a SAM3 segmentation network (b). The resulting RGB mask is associated with the depth map to recover a partial 3D point cloud of the cloth (c). A spline is fit to the observed boundary and extended to account for occluded portions, ensuring that the spline’s length is consistent with the known cloth length (d). Grid points are then fit to the point cloud and points in occluded regions are generated according to the spline (e). This results in a complete cloth state representation that is provided to the simulator (f).
Figure 5: Perception pipeline for cloth state estimate. Raw front-camera images (a) are processed by a SAM3 segmentation network (b). The resulting RGB mask is associated with the depth map to recover a partial 3D point cloud of the cloth (c). A spline is fit to the observed boundary and extended to account for occluded portions, ensuring that the spline’s length is consistent with the known cloth length (d). Grid points are then fit to the point cloud and points in occluded regions are generated according to the spline (e). This results in a complete cloth state representation that is provided to the simulator (f).
Original mathematical definitions, assumptions and evaluation protocol

We use an approximately 38 cm x 38 cm cloth. For our simulation model, the cloth is discretized into a 7 x 7 grid of cells, corresponding to an 8 x 8 lattice of nodes and yielding 64 cloth nodes in total. The full state has dimension (64×3+3×2=19864\times 3+3\times 2=198), consisting of the 3D positions of the cloth nodes and two 3D gripper positions.

Appendix C RL training convergence

APG reaches PPO's cost with mean 45.95% fewer steps for lifting and 16.63% fewer for dragging; reported medians 36.01/27.43% use a different summary. These are training-curve cost-matching comparisons, not faster online policy inference or guaranteed safe learned behavior. The main table's fixed interaction budgets remain alongside the convergence plots and their uncertainty bands.

Complete original Figure 6: RL training convergence for the (a) Lift Rope task, and (b) Drag Rope task.
Figure 6: RL training convergence for the (a) Lift Rope task, and (b) Drag Rope task.
Original selectable panel captions and notation

(a)

(b)

Appendix D State Feedback MPC Tubes

Figure 7 retains all 66 rope-state plots and Figure 8 the full cloth z-coordinate grid. The text explicitly warns that repeatedly replanned execution can leave tubes computed at earlier steps: the controller and future path have changed. Keeping a previously synthesized robust feedback controller gives containment over its own finite horizon, while guaranteed robust recursive feasibility needs an additional approach such as the cited disturbance-feedback MPC. Replanning alone is not a proof that every old tube remains valid. The cloth example's 3 cm height constraint is the particular task constraint, not a global guarantee for arbitrary folding.

Complete original Figure 7: Forward reachable tubes under MPC for the drag rope task for all 66 states.
Figure 7: Forward reachable tubes under MPC for the drag rope task for all 66 states.
Original mathematical definitions, assumptions and evaluation protocol

Fig. 8 shows the forward reachable tubes for the fold cloth task in Fig. 2(c) at selected MPC steps. A constraint limits the cloth height to 3 cm in the zz direction. Because MPC replans after each execution step, the executed trajectory does not necessarily remain within tubes computed at previous time steps. Nevertheless, by enforcing constraints on the tube-valued predictions during each replan, the controller robustly satisfies the zz-height constraint throughout the task.

Complete original Figure 8: Forward reachable tubes under MPC for the fold cloth task for the z z -coordinate states.
Figure 8: Forward reachable tubes under MPC for the fold cloth task for the zz-coordinate states.

Appendix E Output Feedback Tubes

The complete 39-state output-feedback grid shows rollouts using the synthesized observer-based controller under randomized dynamics/measurement disturbances. These fixed-controller finite-horizon plots should be read separately from AppendixD's receding-horizon overlays. Colors denote coordinate directions; the native vector export preserves every bound, label and curve.

Complete original Figure 9: Forward reachable tubes for the output-feedback rope manipulation task for all 39 states. The red trajectories represent the states corresponding to the x x dimension, green the y y dimension, and blue the z z dimension.
Figure 9: Forward reachable tubes for the output-feedback rope manipulation task for all 39 states. The red trajectories represent the states corresponding to the xx dimension, green the yy dimension, and blue the zz dimension.

Appendix F Simulator Comparison

Runtime Table 2 uses 20-node rope/100-node cloth and favors the GPU-native implementation, while the compared DEFORM pipeline is not compiled into GPU-native code. Accuracy Table 3 uses 100 settled hardware transitions,70/30 residual-model train/validation split and 10 links. Latency prevents observing transient swinging; DEFORM is advanced to equilibrium before comparison. Raw mean errors are.0150 m CORD-SLS versus.0139 m DEFORM, and residual-corrected.0075 versus.0071 m. Identical residual architectures reduce one confound, but the small settled-data study does not validate dynamic accuracy. Stretch/bend-only modeling differs from DEFORM's twisting mechanics, and finite differences in runtime/accuracy do not establish universal dominance.

Complete original Table 2: Comparison of simulation time across different differentiable simulators.
Table 2: Comparison of simulation time across different differentiable simulators.
Accessible original table, complete values and grouped headers
Forward simulation [ms] Dynamics Jacobian computation [ms]
Rope Ours 1.095 ±\pm 0.162 5.198 ±\pm 2.533
DEFORM [9] 49.03 ±\pm 20.04 2128.81 ±\pm 55.14
DaXBench [36] 786.48 ±\pm 76.23 11561.57 ±\pm 122.07
Cloth Ours 15.08 ±\pm 1.209 10.06 ±\pm 0.983
DaXBench [36] 21.71 ±\pm 0.722 110.19 ±\pm 2.663
Original mathematical definitions, assumptions and evaluation protocol

We next benchmark the simulation accuracy of our method against DEFORM. To construct the benchmark dataset, we collect 100 samples of randomized manipulation actions on hardware while running our perception pipeline to estimate the rope state. For each sample, we record the current state, control action, and resulting next state, i.e. (x+=f⁡(x,u)x^{+}=f(x,u)). We note that due to the latency in the perception pipeline, we are unable to directly observe the instantaneous post-action state and therefore cannot reliably capture transient swinging or momentum-driven dynamics. Instead, the recorded next state corresponds to the settled configuration of the rope after the motion has dissipated, making the dataset more representative of quasi-static behavior.

Complete original Table 3: Comparison of simulation accuracy of our method against DEFORM.
Table 3: Comparison of simulation accuracy of our method against DEFORM.
Accessible original table, complete values and grouped headers
Mean L2 Error [m]
Rope Ours 0.0150 ±\pm 0.0040
DEFORM [9] 0.0139 ±\pm 0.0029
Ours with learned Residual 0.0075 ±\pm 0.0068
DEFORM with learned Residual [9] 0.0071 ±\pm 0.0032
Concept demonstration

Open image ↗