Overview
Project page. This edition follows the complete twenty-two-page, 12 August 2026 v2, including thirteen figures, eleven numbered tables, all equations and 51 references.

At a glance
FACTR 2 combines two components. NEXT learns the motor torque needed for contact-free motion from a short proprioceptive history; subtracting this prediction from measured current-derived torque supplies an external-torque estimate. FIRST uses that estimate to identify and preferentially sample contact-relevant demonstration segments. Experiments support useful force feedback and improved contact-rich policy learning, with robot-specific modeling, torque-scale calibration and limited contact ground truth qualifying the claims.
1 Introduction
Commodity arms often expose motor current but lack dedicated force sensors. NEXT learns their free-motion behavior from ten minutes of contact-free trajectories, then trains in about one minute. The residual enables force-feedback teleoperation without adding a dedicated sensor and extends FACTR to more robot platforms.
FIRST targets a different problem: sparse alignment and interaction events can be underrepresented during behavior cloning. It changes their sampling frequency while retaining the policy architecture. The reported demonstrations on Piper and other arms support adoption beyond sensorized systems, but do not establish accurate sensing on every robot or under arbitrary hardware changes.
2 Related Works
2.1 Sensorless External Force Estimation
Analytic force observers subtract modeled dynamics from measured effort, making friction, dead zones and transmission effects important. Supervised contact models need labeled interaction data; anomaly detectors mainly identify departures from free motion. Learned actuator models and da Vinci inverse-dynamics models motivate NEXT’s history-conditioned regression on inexpensive arms.
2.2 Imitation Learning with Force Information
TA-VLA, BiACT and CompACT use effort/force information; ForceVLA changes modality routing, while FoAR adds contact-aware objectives. Other methods combine learned motion with force or impedance control. FIRST’s distinction is using inferred contact phases to alter demonstration sampling, with pre-contact alignment often more valuable than sustained contact alone.
2.3 Policy Learning with Dataset Reweighting
Data weighting can emphasize useful actions or demonstrations: AWR, CRR and AWAC use value-related signals; Re-Mix, DataMIL and DemInf optimize dataset selection. SARM, Demo-SCORE and CUPID provide other relevance signals. FIRST instead uses a lightweight physical-contact proxy to weight individual time steps.
3 Background
Motor torque is approximated by measured current times a torque constant. Subtracting free-space dynamics gives the paper’s external joint-torque convention. A rigid-body model must account for inertia, velocity effects, gravity and other actuation terms.
The same residual also contains modeling error: friction, hysteresis, temperature effects, saturation and sensing noise can resemble contact. Temporal context can improve that separation but does not make it identifiable or exact under every operating condition. Joint torque is also distinct from a uniquely localized Cartesian contact force.
Equation 1
| (1) |
4 Learning External Force Estimation
NEXT regresses free-space motor torque from recent joint positions, velocities and commanded-position tracking errors. Contact-free measured torque supplies its own supervision; no external force label is needed for training. At deployment, subtracting the predicted torque from current-derived torque gives the external estimate.
A stateless LSTM processes fifty samples, roughly half a second at 100 Hz, with its hidden state reset for each window. History captures local actuator and friction effects without explicitly estimating acceleration. Training uses about ten minutes of free motion and about one minute on an RTX 3090. This replaces manual analytic identification with learned, robot-specific calibration; it does not remove dependence on the arm, controller or current-to-torque scale.
Equations 2–3
| (2) |
| (3) |

Equation 4
| (4) |

Equation
5 Force-Informed Re-Sampling Training
The policy receives camera images, joint state and NEXT torque, then predicts a chunk of future joint targets through flow matching. Equation 5 retains the velocity-regression objective; Appendix B also reports an ACT implementation.
FIRST uses the L1 norm of estimated external torque as a contact score. Two hysteresis thresholds suppress rapid switching. Contact takes priority; the one-second interval before each contact onset becomes pre-contact, and remaining samples are free space. These are offline labels derived from demonstrations, so future contact onset is available during labeling without requiring future information at deployment.
Sampling probability is proportional to a phase weight. The selected recipe is usually 1:5:1 for free/pre-contact/contact, emphasizing pre-contact alone; cap screwing uses 1:3:3. Thus the actual experiments do not always up-sample both phases despite the broad method description. Numeric hysteresis thresholds are not specified in the paper’s settings tables. The weights are selected using task progress rather than established as a universal default.
Equations 5–7
| (5) |
| (6) |
| (7) |
6 Experiments
6.1 External Force Estimation and Force-Feedback Teleoperation
On Franka, five held-out minutes of human-applied contact are compared with its factory-calibrated external-torque estimate. A separate five-minute free-motion test uses zero external torque as reference. FILIC-style inverse dynamics and a momentum disturbance observer are baselines. Franka’s built-in estimate is a useful reference, not an independent exact ground-truth measurement.
Piper supplies no contact torque reference, so quantitative accuracy is evaluated only in free space. A twenty-participant wiping study tests whether feedback is useful: no feedback, disturbance observer, position-position feedback, sensor-based FACTR and NEXT-based FACTR on Franka; Piper omits the sensor-based condition. Ratings use a five-point ease scale. Measured applied torque and subjective comfort are complementary evidence, not a calibrated contact-force accuracy test.

Equation 8
| (8) |
6.2 Imitation Learning with Force Input
Five long-horizon tasks use 250 demonstrations each on a bimanual Piper setup with four RGB cameras. Each policy receives twenty rollouts per task; the main metric is stage-based task progress, not simply complete-episode success. Baselines hold the policy architecture and cloning objective fixed while changing force input, visual curriculum or an auxiliary torque target.
The flow policy adapts Multimodal Diffusion Transformer; experiments also use ACT. Action chunks contain thirty targets and action heads start from scratch. Base Policy omits force; Base+Torque adds NEXT; FACTR applies a blur curriculum; TA-VLA adds torque reconstruction; FIRST adds phase sampling. These are controlled adaptations within this implementation, not comparisons of every original published system.

7 Results
NEXT’s Franka contact error is 0.547±0.348 Nm, versus FILIC 4.395±1.531 and disturbance observer 1.471±0.761: reported reductions of 87.6% and 62.8%. In free motion it gives 0.414±0.278 Nm versus the built-in estimate’s 0.449±0.208, a modest difference against the zero reference rather than proof of universally better sensing. Piper free-space error is 0.018±0.012 Nm, with no corresponding instrumented contact test.
The user-study plots favor NEXT for ease and reduced applied torque, with Franka behavior similar to sensor-based FACTR. Those results demonstrate useful feedback on the tested wiping setup; the paper does not provide a formal equivalence test against the sensor-based system.
FIRST has the highest reported flow-policy progress on all five tasks. The phase ablation gives average progress 0.670 for contact-only, 0.818 for pre-contact-only and 0.811 for both; cap screwing favors both at 0.825, while pre-contact-only gives 0.673. Gains concern this progress metric; the broad “over 17%” claim should not be read as seventeen additional successful episodes per hundred on every task.
For NIST Belt, pre-contact validation loss is much larger under ordinary sampling and drops under FIRST. This is consistent with focusing learning on alignment, but the loss curves and selected recovery examples do not independently prove a causal mechanism. Raw current helps some tasks; the dedicated residual gives stronger scores on both tested input-ablation tasks.
8 Conclusion
Learning free-space effort makes force-related information usable on sensorless commodity arms, and resampling the moments before contact improves the tested policies. The practical contribution combines a small estimator, force-feedback data collection and a simple training distribution change; accuracy and transfer remain dependent on operating conditions and evaluation scope.
9 Limitations
Absolute torque scale depends on the motor constant K. If it is wrong, physical calibration may still require an external sensor; input normalization or comfortable feedback scaling does not certify accurate Newton-meter measurements. NEXT is robot-specific and may need retraining between arms. The paper does not establish robustness to arbitrary payload, temperature or controller changes, and no safety guarantee follows from a low average estimation error.
Acknowledgments
The authors acknowledge discussion and equipment support, including access to YAM arms, and funding from AFOSR and ONR MURI programs.
Appendix
Appendix A NEXT Implementation Details
A.1 Free-Space Inverse Dynamics Model Architecture
The estimator uses a two-layer LSTM of width 128 and a two-layer prediction MLP of width 256, dropout 0.1 and squared-error supervision. AdamW uses learning rate 10⁻³, weight decay 10⁻⁶ and gradient clipping 1.0; validation-based early stopping controls training.
A forward pass takes about 1.76 ms on one i9-10920X CPU thread, equivalent to 568 evaluations per second, but the deployed loop runs at 100 Hz. That microbenchmark is not end-to-end sensing/communication latency. Resetting recurrent state per rolling window avoids accumulation over a full trajectory.
Original selectable table
| Hyperparameter | Value |
| Recurrent backbone | 2-layer LSTM with hidden size 128 |
| Prediction head | 2-layer MLP with hidden size 256 |
| Dropout | 0.1 |
| Loss function | Mean squared error (MSE) |
| Optimizer | AdamW |
| Base learning rate | |
| Weight decay | |
| Gradient clipping | 1.0 |
| Early stopping | Enabled; validation loss, patience 20, warmup 10, |
| Hardware | NVIDIA RTX 3090, 24 GB |
| Training time | Approximately 1 minute with early stopping |
| Data frequency | 100Hz |
A.2 Free-Space Inverse Dynamics Network Ablations
Table 4 varies MLP/GRU/LSTM, inputs and histories of ten, twenty-five and fifty samples. The chosen full-input fifty-step LSTM gives the best reported free-space/contact pair, 0.414/0.547 Nm.
Adding tracking error or lengthening history is not uniformly better in every configuration: the full-input GRU’s contact errors exceed its position/velocity variant, for example. The table supports the selected configuration rather than the stronger claim that each added modality consistently improves every model. Contact reference and free-space zero targets retain the distinction used in the main evaluation.
A.3 Free-Space Data Collection Guidelines for NEXT
The ten-minute collection procedure first exercises joints independently, then uses coordinated Cartesian-like motion, and repeats both slowly and quickly. Figure 8 preserves all four six-joint distributions: tracking error, position, velocity and free-space torque. Recorded motions can be replayed on another setup to collect new calibration data; this is not reuse of the same estimator without retraining.

A.4 External Joint Torque Estimation Baseline Implementation
Franka baselines use identified dynamics; Piper uses the manufacturer URDF. The FILIC-style residual subtracts modeled inverse dynamics, while the momentum observer avoids explicit acceleration estimation and smooths encoder noise. These baseline model choices affect the comparison with a model fit directly to each arm’s data.
The printed observer equations use a different torque sign from Equation 1’s residual convention: with the stated momentum error and positive observer gain, a constant external residual produces its negative at steady state. The original equations remain unchanged. This is a source notation discrepancy, not evidence that the uninspected baseline implementation used the wrong sign.
Equations 9–11
| (9) |
| (10) |
| (11) |
A.5 Force-Feedback Teleoperation Baseline Implementation
The no-feedback condition uses GELLO. Disturbance-observer feedback sends estimated follower torque to the leader; position-position feedback scales leader/follower tracking error. Sensor-based and NEXT-based FACTR otherwise share gravity/friction compensation and null-space regulation. Piper has four conditions because it lacks dedicated joint-torque sensing.

A.6 AgileX Piper NEXT Evaluations
Piper’s free-space error is 0.018±0.012 Nm, versus FILIC 0.435±0.225 and disturbance observer 0.248±0.132. The separate ease-of-use plot favors NEXT feedback in contact. Near-zero output during free motion is necessary but insufficient to establish its absolute contact accuracy; that remains unmeasured on this platform.
Appendix B Behavior Cloning Policy Architecture and Training Hyperparameters
DINOv3-Base encodes images; MLPs embed joint/gripper state and force. Policies use the latest observation, while NEXT separately supplies its fifty-step history-derived estimate. ACT has four encoder and six decoder layers; the flow model has six decoder layers. Both use width 768, eight attention heads, feed-forward width 3,072 and thirty-action chunks.
Table 6 retains all training settings: batch 128, AdamW at 3×10⁻⁴, cosine decay, 500 warmup steps, random resized crops and 12–24 hours on an A6000. It lists 15–20 epochs while the main comparison states twenty. The main formulation samples flow time uniformly, whereas the implementation table specifies Beta(1.0,1.5), with ten Euler inference steps; both source specifications are preserved.
Original selectable table
| Hyperparameter | Value |
| Behavior Policy Training | |
| Optimizer | AdamW |
| Base Learning Rate | |
| Weight Decay | |
| Optimizer Momentum | |
| Batch Size | 128 |
| Learning Rate Schedule | Cosine Decay |
| Total Epochs | 15–20 |
| Warmup Steps | 500 |
| Augmentation | RandomResizeCrop |
| GPU | A6000 (48 GB) |
| Wall-Clock Time | 12–24 hours |
| Action Chunking Transformer Details | |
| # Encoder Layers | 4 |
| # Decoder Layers | 6 |
| # MHSA Heads | 8 |
| Hidden Dim | 768 |
| Feed-Forward Dim | 3072 |
| Dropout | 0.1 |
| Positional Encoding | Sinusoidal |
| Action Chunk | 30 |
| Flow Matching Details | |
| # Decoder Layers | 6 |
| # MHSA Heads | 8 |
| Hidden Dim | 768 |
| Feed-Forward Dim | 3072 |
| Dropout | 0.1 |
| Positional Encoding | Sinusoidal |
| Action Chunk | 30 |
| Eval Steps | 10 |
| Time Sampling | |
| ODE Solver | Euler |
| Noise Distribution | |
Appendix C Behavior Cloning Evaluation Tasks
C.1 Task Description
LEGO uses LEAP Hand V2 to assemble two small blocks onto a larger block. Gripper tasks wrap belts around pulley sets, insert NIST parts, put away a screwdriver and assemble its case, or pick/alignment/screw a cap until tight. The last task also requires stopping instead of continuing to rotate.
C.2 Task Progress Rate Calculation
Progress is the fraction of listed stages completed across rollouts. Table 7 preserves six LEGO stages, seven belt stages and four each for NIST Insertion, Tool Clean Up and Cap Screwing. The insertion score ends with picking up the aluminum part; it does not list its subsequent alignment/insertion. Therefore a high progress score does not by itself certify completion of both inserted assemblies described at task level.
Cap screwing subtracts 0.02 per excess turn after tightening. This modified score differs from an unpenalized fraction of successful stages and from binary success. “Case Assembly” elsewhere refers to the Tool Clean Up task.
Original selectable table
| Task | Stages |
| LEGO | red grasp, red align, red insert, green grasp, green align, green insert |
| NIST Belt | pick orange, align big, align small, push, pick grey, align grey, push |
| NIST Insertion | pick black, align black, insert black, pick aluminum |
| Tool Clean Up | pick driver, align driver, insert driver, assemble box |
| Cap Screwing | pick cap, align cap, screw on, end properly |
C.3 Imitation Learning with Force Information Baseline Implementations
The base visual/proprioceptive policy receives no torque; Base+Torque adds a projected NEXT token. FACTR applies Gaussian image blur that decreases linearly through training. TA-VLA adds a torque/contact prediction target to the action-related representation; only robot-action coordinates execute at inference. The full visual curriculum formula is retained.
Equation
Appendix D Additional Results on FIRST
D.1 Segmenting Trajectories into Contact Phases Visualization
The segmentation example overlays normalized torque, phase bands and images. It illustrates low free-motion residuals and increasing signal around interaction, including the interval immediately before contact. It is a qualitative example rather than a labeled-contact precision/recall study, and segmentation inherits estimator and threshold errors.

D.2 Task Progress Evaluation on ACT
The ACT comparison also favors FIRST across the five plotted task-progress outcomes. This extends the result beyond the flow head within the shared visual encoder/data setup; it does not establish architecture-independent gains under arbitrary policies or larger datasets.
D.3 Selected Re-Sampling Ratios for Each Task
Table 8 selects free/pre-contact/contact weights 1:5:1 for LEGO, NIST Belt, NIST Insertion and Tool Clean Up, and 1:3:3 for Cap Screwing. The weights are chosen by task progress. They change sampling probabilities relative to the dataset’s original phase counts, not the final batch fractions directly.
D.4 Up-Sampling Magnitude Ablation
ACT ablations cover NIST Belt, NIST Insertion and Tool Clean Up; the flow ablation covers only Tool Clean Up. Moderate weighting often helps and aggressive weighting can lose full-trajectory coverage. Around five is a useful reported trend rather than a universal optimum: ACT’s pre-contact-only NIST Belt curve peaks at three, while its combined-phase curve peaks at five.
Equations

D.5 Up-Sampling Policies Without Force Input
NEXT can label demonstrations offline while the deployed policy receives only images and proprioception. Phase sampling still helps in the two-task ablation, so not every use of FIRST requires online force input. Contact information at inference generally adds further gains; some individual sampling comparisons tie rather than improve.
For flow matching on NIST Belt, pre-contact-only sampling rises from 0.150 to 0.213 without torque and from 0.431 to 0.767 with torque. On Tool Clean Up it rises from 0.463 to 0.575 without torque and from 0.525 to 0.805 with torque. Table 9 calls these success rates, whereas the main figures use stage progress; its values and label are retained without treating the two metrics as interchangeable.
Original selectable table
| Policy | Up-sampling Strategy | w/o | w/ | ||
| Belt Assembly | Tool Clean Up | Belt Assembly | Tool Clean Up | ||
| ACT | None | 0.220 | 0.600 | 0.347 | 0.650 |
| PC | 0.360 | 0.600 | 0.687 | 0.650 | |
| PC + C | 0.283 | 0.763 | 0.770 | 0.788 | |
| Flow Matching | None | 0.150 | 0.463 | 0.431 | 0.525 |
| PC | 0.213 | 0.575 | 0.767 | 0.805 | |
| PC + C | 0.241 | 0.525 | 0.649 | 0.775 | |
D.6 Inputting Joint Current vs. External Joint Torque on Policy Performance
ACT’s reported NIST Insertion score is 0.220 without force, 0.431 with raw current and 0.567 with NEXT. Cap Screwing is 0.440/0.440/0.525. A specialized history-based inverse-dynamics model can separate free-motion effort from contact more effectively than supplying only instantaneous current to the policy under this comparison.
Figure 13 plots only first-decoder-layer attention from action tokens to force-related tokens. Torque-conditioned attention increases around tightening, while the current-conditioned policy continues rotating. This supports a diagnostic interpretation; attention alone is not causal attribution or evidence that longer-history current-conditioned policies could not match the result.

D.7 Per-batch Up-sampling vs. Weighted Regression
The separate ACT LEGO comparison gives progress 0.453 with no weighting, 0.345/0.478 with pre-contact/combined-phase weighted regression, and 0.573/0.501 with weighted sampling. Sampling performs better in this experiment; proposed instability/overfitting explanations are not directly established by that table.
For fixed phase weights, sampling proportional to weight and normalized weighted regression have the same expected weighted empirical objective, up to a constant scale. Their minibatch composition, gradient variance, normalization and optimizer dynamics can differ. The result therefore supports the reported sampling implementation, not a fundamental distinction between their expected objectives.