Literature
Display
← All papers

PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations

Yang Zhang · Jiangyuan Zhao · Chenyou Fan · Fangzheng Yan · Tian Li · Haitong Tang · Sen Fu · Xuan’er Wu · Qizhen Weng · Weinan Zhang · Xiu Li · Chi Zhang · Chenjia Bai · Xuelong Li

Preprint posted · CoRL 2026 (accepted) ↗

Robotics11 min text · CondensedOriginal paper ↗
Contents

At a glance

PRTS adds temporally weighted contrastive learning to vision-language-action pretraining. State–action embeddings identify the relevant language goal, while a reverse contrastive objective ranks same-task examples by discounted distance to completion. A role-aware attention mask computes these representations alongside discrete action prediction in one backbone pass. After pretraining on 167.8B tokens, a continuous action expert is attached and fine-tuned. Evaluation covers simulation, 14 physical tasks, controlled shifts and qualitative intervention recovery; the learned scores are evidence of task/progress structure, not calibrated probabilities of arbitrary real-world success.

1 Introduction

The motivating gap is that immediate action imitation and general visual-language supervision do not explicitly teach progress toward a goal. PRTS supplies this signal from demonstration timing and task identity, drawing on contrastive reinforcement learning. It combines a multi-positive temporal objective, isolated state–action and language representations, and an efficient shared forward pass. Training remains offline and demonstration-based: the method adds an RL-inspired representation objective rather than collecting policy-improvement rollouts during pretraining.

2 Preliminaries

For the specified goal-reaching reward, a policy’s discounted return equals discounted future occupancy of the goal. Standard contrastive RL distinguishes future goal samples from marginally sampled goals, learning a log density ratio. Equation 7 includes both a goal-marginal correction and a state–action-dependent offset; raw embedding similarity therefore should not be read as an absolute success probability. This occupancy connection motivates PRTS, whose language goals require a different positive-sampling construction.

Equations 1–7
rg​(st,at)≜(1−γ)​p​(st+1=sg∣st,at),\displaystyle r_{g}(s_{t},a_{t})\triangleq(1-\gamma)\,p(s_{t+1}=s_{g}\mid s_{t},a_{t}), (1)
Qsgπ(s,a)≜𝔼π[∑t′=t∞γt′−trg(st′,at′)|st=s,at=a].\displaystyle Q^{\pi}_{s_{g}}(s,a)\triangleq\mathbb{E}_{\pi}\left[\sum_{t^{\prime}=t}^{\infty}\gamma^{t^{\prime}-t}r_{g}(s_{t^{\prime}},a_{t^{\prime}})\,\middle|\,s_{t}=s,a_{t}=a\right]. (2)
pπ(⋅∣⋅,sg)(st+=s)≜(1−γ)∑t=0∞γtptπ(⋅∣⋅,sg)(st=s),\displaystyle p^{\pi(\cdot\mid\cdot,s_{g})}(s_{t+}=s)\triangleq(1-\gamma)\sum_{t=0}^{\infty}\gamma^{t}p^{\pi(\cdot\mid\cdot,s_{g})}_{t}(s_{t}=s), (3)
Qsgπ(s,a)=pπ(⋅∣⋅,sg)(st+=sg∣s,a).\displaystyle Q^{\pi}_{s_{g}}(s,a)=p^{\pi(\cdot\mid\cdot,s_{g})}(s_{t+}=s_{g}\mid s,a). (4)
f⁡(s,a,sg)=ϕ​(s,a)⊤​ψ​(sg),\displaystyle f(s,a,s_{g})=\phi(s,a)^{\top}\psi(s_{g}), (5)
ℒcrl=maxf⁡𝔼(s,a)∼p⁡(s,a),sg+∼pπ​(st+∣s,a),sg−∼p⁡(sg)​[log⁡exp⁡(f⁡(s,a,sg+))exp⁡(f⁡(s,a,sg+))+∑exp⁡(f⁡(s,a,sg−))],\displaystyle\mathcal{L}_{\rm crl}=\max_{f}\,\,\mathbb{E}_{\begin{subarray}{c}(s,a)\sim p(s,a),\\ s_{g}^{+}\sim p^{\pi}(s_{t+}\mid s,a),\\ s_{g}^{-}\sim p(s_{g})\end{subarray}}\Big[\log\frac{\exp(f(s,a,s_{g}^{+}))}{\exp(f(s,a,s_{g}^{+}))+\sum\exp(f(s,a,s_{g}^{-}))}\Big], (6)
f∗​(s,a,sg)=log⁡Qsgπ​(s,a)−log⁡p⁡(sg)+c⁡(s,a),\displaystyle f^{*}(s,a,s_{g})=\log Q^{\pi}_{s_{g}}(s,a)-\log p(s_{g})+c(s,a), (7)

3 Method

The method adapts contrastive goal learning to task-labelled demonstrations, then integrates it into a backbone that also predicts action tokens. Continuous control is learned in a separate post-training stage.

3.1 Problem Setup

Each demonstration supplies a task instruction, camera images, proprioception and an action sequence. PRTS seeks representations that encode task identity and progress without separately annotated rewards, progress comparisons or a separately trained critic. Task labels and demonstration lengths still provide supervision; “reward-label-free” does not mean learning without structured data or an assumption that the demonstrations approach completion.

3.2 Language-Conditioned Contrastive Reinforcement Learning

All batch samples from the same task are positives for its fixed language goal. Their normalized target weights are proportional to γ raised to the remaining trajectory length, emphasizing examples nearer completion. The state–action-to-language loss primarily distinguishes tasks; merely weighting each example’s classification loss does not impose an ordering within its trajectory. The reverse language-to-state–action loss distributes probability among multiple same-task positives according to those temporal weights, creating the progress signal.

Theorem 1 identifies the ideal reverse-objective similarity with log discounted occupancy plus a goal-dependent constant under deterministic expert demonstrations. Its interpretation relies on the terminal goal and trajectory assumptions made in Appendix B. It does not establish calibrated reachability for stochastic, failed or out-of-distribution behavior. Action chunks use the same construction with a chunk as the action representation.

Equations 8–13
𝒮⁡(i)≜{j∈{1,…,B}:τj=τi}.\displaystyle\mathcal{S}(i)\triangleq\{j\in\{1,\ldots,B\}:\tau_{j}=\tau_{i}\}. (8)
f∗​(s,a,sg)=log⁡pπ​(st+=sg∣s,a)p⁡(sg)+c⁡(s,a),\displaystyle f^{*}(s,a,s_{g})=\log\frac{p^{\pi}(s_{t+}=s_{g}\mid s,a)}{p(s_{g})}+c(s,a), (9)
qi​j=γTj−tj∑j′∈𝒮⁡(i)γTj′−tj′.\displaystyle q_{ij}=\frac{\gamma^{T_{j}-t_{j}}}{\sum_{j^{\prime}\in\mathcal{S}(i)}\gamma^{T_{j^{\prime}}-t_{j^{\prime}}}}. (10)
ℒsa→l=𝔼i∼ℬ​[−γTi−ti​log⁡exp⁡(ϕi⊤​ψi)exp⁡(ϕi⊤​ψi)+∑k∈ℬ,τk≠τiexp⁡(ϕi⊤​ψk)],\displaystyle\mathcal{L}^{\text{sa}\to l}=\mathbb{E}_{i\sim\mathcal{B}}\left[-\gamma^{T_{i}-t_{i}}\log\frac{\exp(\phi_{i}^{\top}\psi_{i})}{\exp(\phi_{i}^{\top}\psi_{i})+\sum_{k\in\mathcal{B},\,\tau_{k}\neq\tau_{i}}\exp(\phi_{i}^{\top}\psi_{k})}\right], (11)
ℒl→sa=𝔼i∼ℬ[−∑j∈𝒮⁡(i)qi​jlogexp⁡(ψi⊤​ϕj)∑k∈ℬexp⁡(ψi⊤​ϕk)],withqi​j=γTj−tj∑j′∈𝒮⁡(i)γTj′−tj′.\displaystyle\mathcal{L}^{l\to\text{sa}}=\mathbb{E}_{i\sim\mathcal{B}}\left[-\sum_{j\in\mathcal{S}(i)}q_{ij}\log\frac{\exp(\psi_{i}^{\top}\phi_{j})}{\sum_{k\in\mathcal{B}}\exp(\psi_{i}^{\top}\phi_{k})}\right],\quad\text{with}\quad q_{ij}=\frac{\gamma^{T_{j}-t_{j}}}{\sum_{j^{\prime}\in\mathcal{S}(i)}\gamma^{T_{j^{\prime}}-t_{j^{\prime}}}}. (12)
ψ∗​(l)⊤​ϕ∗​(s,a)=log⁡Qlπ∗​(s,a)+C⁡(l),\displaystyle\psi^{*}(l)^{\top}\phi^{*}(s,a)=\log Q^{\pi^{*}}_{l}(s,a)+C(l), (13)

3.3 PRTS Architecture

The Qwen3-VL backbone receives visual tokens, discretized textual robot state, instruction tokens and FAST action tokens. Two appended blocks repeat the action and instruction, ending in learned pooling tokens. The action-contrastive block can attend to vision, state and itself, but cannot read the language or autoregressive-action stream. The goal block attends only within its own instruction block. Ordinary action tokens retain causal attention. These restrictions prevent shortcut leakage between the two embeddings while preserving the usual action-prediction computation.

Small projection heads, normalization and a learned temperature produce contrastive scores; cross-device negatives enlarge the batch pool. Pretraining combines action-token, robot-QA, subtask and bidirectional contrastive losses. A custom CuTe attention kernel and sequence packing reduce overhead. Post-training adds a randomly initialized flow-matching DiT expert and uses only its continuous-action loss, with gradients also updating the backbone. The expert uses five denoising steps. The paper demonstrates representation shaping rather than an explicit deployment-time value-search procedure.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 1: Overview of PRTS. PRTS is built on Qwen3-VL-4B-Instruct and involves two training stages. (i) In the pre-training stage, the backbone consumes a unified token sequence of multi-view visual tokens (Qwen ViT), textual proprioceptive-state tokens, language-instruction tokens, and FAST-tokenized AR action tokens, followed by two auxiliary blocks <CRL_action> and <CRL_goal>, in which the last learnable tokens <|action_end|> and <|goal_end|> are passed through lightweight MLP heads (right) to produce the state-action representation ϕ⁡(st,𝐚t)\phi(s_{t},\mathbf{a}_{t}) and the goal representation ψ⁡(l)\psi(l). A role-aware causal mask (bottom-left) prevents information leakage across the five token roles: <CRL_action> tokens attend only to vision and proprioception; <CRL_goal> tokens attend only within their own block; AR action tokens keep standard causal attention. All three outputs—discrete-action logits, ϕ⁡(st,𝐚t)\phi(s_{t},\mathbf{a}_{t}), and ψ⁡(l)\psi(l)—are produced in a single forward pass and trained jointly under the bidirectional contrastive (InfoNCE) losses with temporal weighting and the next-token cross-entropy loss, so that ϕ​(st,𝐚t)⊤​ψ​(l)≈log⁡Qlπ​(st,𝐚t)\phi(s_{t},\mathbf{a}_{t})^{\top}\psi(l)\approx\log Q^{\pi}_{l}(s_{t},\mathbf{a}_{t}). The action expert is absent at this stage. (ii) In the post-training stage, a freshly initialized DiT-based flow-matching action head is attached and fine-tuned via ℒFM\mathcal{L}_{\text{FM}} to produce continuous action chunks.
Equations 14–17
ℒFM=𝔼τ∼[0,1],ϵ∼𝒩⁡(0,I)​‖fθ​(𝐚tτ,st,l)−(𝐚t−ϵ)‖22,\displaystyle\mathcal{L}_{\text{FM}}=\mathbb{E}_{\tau\sim[0,1],\,\epsilon\sim\mathcal{N}(0,I)}\left\|f_{\theta}\!\left(\mathbf{a}^{\tau}_{t},\,s_{t},\,l\right)-(\mathbf{a}_{t}-\epsilon)\right\|_{2}^{2}, (14)
ℒpre-train=ℒBC+ℒRobotQA+ℒSubtask+λcrl​(ℒsa→l+ℒl→sa),\displaystyle\mathcal{L}_{\text{pre-train}}=\mathcal{L}_{\text{BC}}+\mathcal{L}_{\text{RobotQA}}+\mathcal{L}_{\text{Subtask}}+\lambda_{\text{crl}}\left(\mathcal{L}^{\text{sa}\to l}+\mathcal{L}^{l\to\text{sa}}\right), (15)
ℒpost-train=ℒFM.\displaystyle\mathcal{L}_{\text{post-train}}=\mathcal{L}_{\text{FM}}. (16)
Qlπ​(s,𝐚)∝exp⁡(ϕ​(s,𝐚)⊤​ψ​(l)),\displaystyle Q^{\pi}_{l}(s,\mathbf{a})\propto\exp\left(\phi(s,\mathbf{a})^{\top}\psi(l)\right), (17)

4 Datasets

The reported corpus contains 404M samples and 167.8B tokens, divided between action-labelled robot experience and visual reasoning. The dataset diagram gives the original token-share breakdown.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 2: Pre-training dataset overview of PRTS. Our pre-training dataset is partitioned into two primary domains: Action-Labeled Data provides physical execution priors for cross-embodiment manipulation, while Visual-Reasoning Data provides complementary supervision for spatial grounding, affordance perception, and task planning. The area of each segment is scaled proportionally to the square root of its total tokens to highlight the rich, structural diversity of our data sources.

4.1 Action-Labeled Data

Action supervision combines AgiBotWorld, RoboMind, Open X-Embodiment and internally collected RealMan/Agilex trajectories. The sources broaden task types, robot hardware, contact-rich manipulation and long-horizon coordination; some also provide fine-grained subtask labels. Their diversity supports pretraining, while downstream evaluation still uses task- and platform-specific demonstrations.

4.2 Visual-Reasoning Data

Grounding data cover referring expressions, pointing, spatial relations and affordances through RefCOCO, Pixmo-Point, RoboPoint, RoboRefIt, RefSpatial and RoboAfford. Cosmos-Reason1, EgoPlanIT and RoboVQA add physical/temporal reasoning and subgoal prediction. A curated LLaVA-Instruct subset preserves general vision-language competence. These complementary objectives mean the full model’s capabilities cannot be attributed to the contrastive loss alone; the matched ablation tests its incremental effect.

5 Related Work

PRTS builds on discrete-action VLAs and later systems that combine semantic supervision with continuous action experts. Its distinction from separate value modules is that the contrastive objective directly updates the shared vision-language representation. Contrastive RL supplies a classification-based alternative to Bellman or Monte Carlo regression; PRTS adapts that idea to fixed language goals and multiple timed positives. Claims about advantages over other training families should be separated from the narrower evidence of its own controlled ablation.

6 Experiments

Experiments examine nominal success, the contribution of contrastive pretraining, distribution shifts, instruction recombination, recovery after interventions, learned score structure and training efficiency. Simulation tables use benchmark-specific baseline sets; the physical comparison uses π₀ and π₀.₅.

6.1 Experimental Setup

Pretraining starts from Qwen3-VL-4B-Instruct with contrastive coefficient 1 and temporal discount 0.995. It uses packed length-4096 sequences, global batch 256, roughly 2K contrastive negatives, about 220K updates and one week on 64 H100s. The 675M-parameter action expert is attached afterward. Post-training schedules vary: LIBERO uses batch 32 for 30K steps and horizon 20; SimplerEnv uses 1024/20K/horizon 16; RealMan uses 64/100K/horizon 20; Flexiv uses 32/40K/horizon 20.

RealMan is a 14-DoF dual-arm platform with head and wrist cameras; Flexiv provides single-arm tasks. Each physical task cell has 20 trials. Flexiv permits up to five grasp attempts or two minutes; RealMan grasping tasks permit five feasible retries and a three-minute limit. Reported batch-times-step budgets make data exposure visible, but are not normalized total FLOPs across different backbones or pretraining corpora.

Benchmark Global batch size Gradient steps Action chunk HH
LIBERO 3232 3030K 2020
SimplerEnv (WidowX) 10241024 2020K 1616
RealMan Dual-Arm Robot Platform (all-task fine-tuning) 6464 100100K 2020
Flexiv Robot Platform (single-task fine-tuning) 3232 4040K 2020
Table 1: Post-training recipes per benchmark.
Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 3: Overview of real-world robot platforms. Left: dual-arm RealMan platform with two parallel-jaw grippers, one egocentric head camera, and two wrist cameras (one per arm). Right: Flexiv single-arm platform equipped with a parallel-jaw gripper, a third-view head camera, and two wrist cameras.

6.2 Overall Performance

The standard-LIBERO checkpoint is evaluated directly on LIBERO-Plus and LIBERO-Pro, with no additional perturbation-specific training. SimplerEnv uses separate post-training on real BridgeData V2 trajectories. These are downstream adaptation and transfer results, rather than an entirely unadapted pretrained policy evaluation.

6.2.1 LIBERO

PRTS reaches 98.4% average LIBERO success and 96.6% on Long. At the same batch-32/30K-step budget, ABot-M0 reaches 97.9% average and 94.8% Long; its larger-budget reported model reaches 98.6% average. PRTS therefore approaches the best reported mean with less post-training exposure, rather than exceeding every baseline. The 0.5-point matched-budget advantage is small on this near-saturated benchmark.

Method Spatial Object Goal Long Average
Diffusion Policy (Chi et al., 2023) 78.5 87.5 73.5 64.8 76.1
OpenVLA (Kim et al., 2024) 84.7 88.4 79.2 53.7 76.5
SpatialVLA (Qu et al., 2025) 88.2 89.9 78.6 55.5 78.1
CoT-VLA (Zhao et al., 2025) 87.5 91.6 87.6 69.0 83.9
GR00T-N1 (Bjorck et al., 2025) 94.4 97.6 93.0 90.6 93.9
F1-VLA (Lv et al., 2025) 98.2 97.8 95.4 91.3 95.7
InternVLA-M1 (Chen et al., 2025) 98.0 99.0 93.8 92.6 95.9
Discrete Diffusion VLA (Liang et al., 2025) 97.2 98.6 97.4 92.0 96.3
GR00T-N1.6 (Bjorck et al., 2025) 97.7 98.5 97.5 94.4 97.0
OpenVLA-OFT (Kim et al., 2025) 97.6 98.4 97.9 94.5 97.1
X-VLA (Zheng et al., 2025) 98.2 98.6 97.8 97.6 98.1
π0\pi_{0}-Fast (Pertsch et al., 2025) (bs=32, 30K steps) 96.4 96.8 88.6 60.2 85.5
π0\pi_{0} (Black et al., 2024) (bs=32, 30K steps) 96.8 98.8 95.8 85.2 94.2
Qwen3-VL-PI (starVLA Contributors, 2025) (bs=32, 30K steps) 95.2 99.0 96.2 88.4 94.7
π0.5\pi_{0.5} (Black et al., 2025) (bs=256, 30K steps) 98.8 98.2 98.0 92.4 96.9
ABot-M0 (Yang et al., 2026) (bs=32, 30K steps) 98.6 99.6 98.6 94.8 97.9
ABot-M0 (Yang et al., 2026) (bs=64, 40K steps) 98.8 99.8 99.0 96.6 98.6
PRTS (Ours) (bs=32, 30K steps) 98.8 99.8 98.4 96.6 98.4
Table 2: Evaluation results on the LIBERO benchmark. Post-training budget is indicated in gray next to the model names. Best results are in bold, and second-best results are underlined. PRTS matches the strongest prior results with global batch size 3232 and 3030K post-training steps; ABot-M0 under the same budget is included for a controlled comparison.

6.2.2 LIBERO-Plus

On LIBERO-Plus, PRTS averages 81.4% versus π₀.₅’s 80.7%. Gains are concentrated in robot-state and background shifts, while π₀.₅ is stronger on some other axes, including noise and layout. This supports an aggregate robustness improvement, not uniformly dominant performance under every perturbation.

Method Camera Robot Language Light Background Noise Layout Average
OpenVLA (Kim et al., 2024) 0.8 3.5 23.0 8.1 34.8 15.2 28.5 15.6
WorldVLA (Cen et al., 2025) 0.1 27.9 41.6 43.7 17.1 10.9 38.0 25.0
NORA (Hung et al., 2025) 2.2 37.0 65.1 45.7 58.6 12.8 62.1 39.0
UniVLA (Bu et al., 2025) 1.8 46.2 69.6 69.0 81.0 21.2 31.9 42.9
RIPT-VLA (Tan et al., 2025) 55.2 31.2 77.6 88.4 91.6 73.5 74.2 68.4
OpenVLA-OFT (Kim et al., 2025) 56.4 31.9 79.5 88.7 93.3 75.8 74.2 69.6
π0\pi_{0} (Black et al., 2024) (bs=32, 30K steps) 13.8 6.0 58.8 85.0 81.4 79.0 68.9 53.6
π0\pi_{0}-Fast (Pertsch et al., 2025) (bs=32, 30K steps) 65.1 21.6 61.0 73.2 73.2 74.4 68.8 61.6
Qwen3-VL-PI (starVLA Contributors, 2025) (bs=32, 30K steps) 43.6 46.6 81.8 88.4 89.4 70.1 74.7 68.9
ABot-M0 (Yang et al., 2026) (bs=32, 30K steps) 65.7 53.1 81.2 96.8 95.0 87.5 81.3 78.7
π0.5\pi_{0.5} (Black et al., 2025) (bs=256, 30K steps) 64.0 58.0 88.5 96.6 81.4 87.5 85.9 80.7
PRTS (Ours) (bs=32, 30K steps) 59.5 72.4 89.6 96.5 96.7 81.3 83.3 81.4
Table 3: Evaluation results on the LIBERO-Plus benchmark. Zero-shot robustness evaluation of the LIBERO-finetuned checkpoint across seven perturbation axes (Fei et al., 2025). Post-training budget is indicated in gray when available. Best results are in bold, and second-best results are underlined.

6.2.3 LIBERO-Pro

LIBERO-Pro averages 58.8%, compared with 53.3% for π₀.₅. PRTS is strongest on Position at 24.3% and novel Task instructions at 31.5%; the latter compares with 0.8% for π₀.₅ and 22.3% for ABot-M0. Object/semantic axes are easier and are not all led by PRTS. Environment perturbations were omitted because their official assets were unavailable. The low absolute success on Position and Task remains a substantial limitation despite the relative gains.

Method Semantic Object Position Task Average
OpenVLA (Kim et al., 2024) 97.2 93.0 0.0 0.0 47.6
Qwen3-VL-PI (starVLA Contributors, 2025) (bs=32, 30K steps) 95.5 70.4 4.3 3.9 43.5
ABot-M0 (Yang et al., 2026) (bs=32, 30K steps) 97.1 82.5 7.1 22.3 52.2
π0\pi_{0} (Black et al., 2024) (bs=32, 30K steps) 90.5 90.5 0.0 0.0 45.3
π0.5\pi_{0.5} (Black et al., 2025) (bs=256, 30K steps) 95.8 96.0 20.8 0.8 53.3
PRTS (Ours) (bs=32, 30K steps) 97.0 82.3 24.3 31.5 58.8
Table 4: Evaluation results on the LIBERO-Pro benchmark. Zero-shot generalization evaluation of the LIBERO-finetuned checkpoint across semantic, object, position, and task variations (Zhou et al., 2025b). Post-training budget is indicated in gray when available. Best results are in bold, and second-best results are underlined.

6.2.4 SimplerEnv with Visual Matching

On SimplerEnv’s WidowX visual-matching tasks, PRTS reaches 77.1% average versus GR00T-N1.5’s 61.9% under the stated matching post-training batch and update count. Scores are 83.3% for spoon placement, 66.6% for carrot placement, 58.3% for block stacking and 100% for eggplant placement. This demonstrates adaptation to another robot/data setting; it does not remove the BridgeData fine-tuning requirement.

Method Put Spoon On Towel Put Carrot On Plate Stack Green Block On Yellow Block Put Eggplant In Yellow Basket Average
OpenVLA (Kim et al., 2024) 0.0 0.0 0.0 4.1 1.0
RT-1-X (Collaboration., 2023) 0.0 4.2 0.0 0.0 1.1
OpenVLA-OFT (Kim et al., 2025) 34.2 30.0 30.0 72.5 41.8
SpatialVLA (Qu et al., 2025) 16.7 25.0 29.2 100.0 42.7
CogACT (Li et al., 2024a) 71.7 50.8 15.0 67.5 51.3
F1-VLA (Lv et al., 2025) 50.0 70.8 50.0 66.7 59.4
Qwen3-VL-PI (starVLA Contributors, 2025) 78.1 46.9 30.2 88.5 60.9
GR00T-N1.5 (Bjorck et al., 2025) 75.3 54.3 57.0 61.3 61.9
π0\pi_{0} (Black et al., 2024) 29.1 0.0 16.6 62.5 27.1
π0.5\pi_{0.5} (Black et al., 2025) 29.2 41.7 0.0 41.7 28.2
π0\pi_{0}-Fast (Pertsch et al., 2025) 29.1 21.9 10.8 66.6 48.3
PRTS (Ours) 83.3 66.6 58.3 100.0 77.1
Table 5: Evaluation results on the SimplerEnv WidowX benchmark (Visual Matching). All numbers are success rates (%). Best results are in bold, and second-best results are underlined.

6.2.5 Real-World Evaluation

A single mixed-task RealMan checkpoint averages 95.9% across eleven tasks, versus 85.5% for π₀.₅ and 67.3% for π₀. All PRTS task scores are at least 90%; the long office sequence improves particularly strongly, from π₀.₅’s 40% to 95%. Flexiv instead uses per-task fine-tuning: PRTS averages 90% versus 75%, with 100% block combination, 70% flower arrangement and 100% gear assembly. These comparisons show strong task performance under the stated retries and time limits; the matched CRL ablation provides the more direct evidence about the objective itself.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
(a) RealMan task suite.
Original figure
(b) Flexiv task suite.
Figure 4: Real-world evaluation. In both suites, per-task success rates averaged over 2020 real-world trials are reported. (a) RealMan dual-arm task suite. PRTS achieves 95.9%95.9\% average SR and reaches at least 90%90\% on all 1111 tasks, outperforming π0.5\pi_{0.5} and π0\pi_{0} under the same evaluation protocol. (b) Flexiv single-arm task suite. PPRTS outperforms the baselines in all three tasks spanning spatial-aligned and fine-grained and contact-rich manipulation.

6.3 Effect of Contrastive RL

Removing CRL while holding PRTS architecture, data and downstream recipe fixed lowers average success from 98.4% to 97.8% on LIBERO, 81.4% to 76.5% on Plus, and 58.8% to 53.8% on Pro. The larger gains occur under shifts: Pro Position improves by 10.5 points and Task by 11.4. Effects are not universal: Object-Task falls from 10.0% to 8.8%, and Pro Object falls by 2.2 points. The accompanying t-SNE visualization shows more separated manipulation-related groups with CRL, but remains a qualitative projection rather than proof of causal compositional reasoning.

Benchmark Suite / Perturbation PRTS w/o CRL PRTS w/ CRL Δ\Delta
LIBERO Spatial 97.8 98.8 +1.0
Object 99.8 99.8 0.0
Goal 98.0 98.4 +0.4
Long 95.6 96.6 +1.0
Average 97.8 98.4 +0.6
LIBERO-Plus Camera 56.7 59.5 +2.8
Robot 56.1 72.4 +16.3
Language 87.3 89.6 +2.3
Light 96.4 96.5 +0.1
Background 95.8 96.7 +0.9
Noise 72.3 81.3 +9.0
Layout 82.9 83.3 +0.4
Average 76.5 81.4 +4.9
LIBERO-Pro Semantic 96.8 97.0 +0.2
Object 84.5 82.3 -2.2
Position 13.8 24.3 +10.5
Task 20.1 31.5 +11.4
Average 53.8 58.8 +5.0
Table 6: Effect of contrastive RL pre-training across the LIBERO benchmark family. Both variants are post-trained on the same standard LIBERO dataset and evaluated zero-shot on LIBERO-Plus and LIBERO-Pro. All numbers are success rates (%), and Δ\Delta reports PRTS w/ CRL minus PRTS w/o CRL.
Method LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-Long
Pos (Avg.) Task (Avg.) Pos (Avg.) Task (Avg.) Pos (Avg.) Task (Avg.) Pos (Avg.) Task (Avg.)
PRTS w/o CRL 26.8 34.4 4.8 10.0 13.0 20.2 10.4 15.6
PRTS w/ CRL 26.8 62.2 36.0 8.8 19.0 33.8 15.4 21.2
Δ\Delta 0.0 +27.8 +31.2 -1.2 +6.0 +13.6 +5.0 +5.6
Table 7: Suite-level performance breakdown on LIBERO-Pro. Success rates (%) are decomposed by the original LIBERO suites to illustrate performance under Position (spatial adaptability) and Task (procedural generalization) distribution shifts. The Δ\Delta denotes PRTS w/ CRL minus PRTS w/o CRL.
Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 5: T-SNE visualizations of learned task representations.

6.4 Controlled Real-World Generalization

Four RealMan tasks are tested under illumination, position, same-category object and instruction changes. PRTS retains 80–100% success under the first three axes. Novel instructions recombine already learned skills: putting a mouse into the receptacle, selecting a soy-sauce bottle among distractors, packing a bottle and cup into the shoebox, and replacing the final cup-stack step with paper disposal. Their respective success rates are 80%, 55%, 85% and 75%, averaging 73.8% versus π₀.₅’s 35.0%. This is meaningful recombination of known primitives, with instruction changes still harder than visual shifts; it is not a test of arbitrary unseen motor skills.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 6: Controlled visual, layout, and object shifts on the RealMan platform. We visualize the three lower-level generalization settings used in the RealMan study. Illumination generalization changes the lighting condition, spatial generalization moves the manipulated objects outside the post-training layout distribution, and object generalization changes the visual instance while preserving the semantic object category and the task instruction. Quantitative results are reported in Table 9.
Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 7: Controlled task-level generalization on the RealMan platform. Task generalization changes the language goal by recombining components of seen tasks, making it semantically stricter than illumination, spatial, or object-level shifts. This setting tests whether the policy can bind a novel instruction to feasible manipulation steps instead of merely recognizing familiar objects or replaying a familiar trajectory. Quantitative results are reported in Table 9.

6.5 Robustness and Recovery under Human Interventions

Intervention examples move objects before grasping, remove them during transport, or reset a completed stage. PRTS is reported to re-localize targets and resume the affected subgoals on Paper Rubbish, Pick Shoes, Place Block and Gear Assembly. The study uses roughly five trials per task and explicitly reports qualitative behavior instead of a formal success-rate table. It illustrates recovery, but does not quantify reliability or establish a broadly validated recovery guarantee.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 8: Robustness and recovery under human interventions. We visualize representative intervention rollouts in which a human repeatedly perturbs task-relevant objects after the policy has committed to an action or subgoal. The traces cover bimanual receptacle interaction, multi-object organization, precise placement, and contact-rich assembly. Successful recovery requires the policy to recognize the changed state, re-estimate task progress, and choose the next action according to the still-unreached language goal.
Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 9: CRL value on a held-out Pick Shoes trajectory, scored by the pre-trained PRTS with no RealMan task suite fine-tuning. Left: the two instructions used to score the same demonstration—a correct one (green, top) that describes what the trajectory actually does, and a wrong one (red, bottom) that swaps the named grasp targets to objects (soy sauce, green paper cup) that never appear in the scene. Top right: five keyframes from the trajectory with their timestep indices, marking the structural milestones of the task (first shoe lifted, first shoe placed, second shoe placed, lid closing, box almost shut). Bottom right: the contrastive value ϕ​(st,𝐚t)⊤​ψ​(l)\phi(s_{t},\mathbf{a}_{t})^{\top}\psi(l) as a function of timestep tt, computed under the correct instruction (green) and the wrong instruction (red) on the same state-action sequence; dashed lines link each keyframe to its corresponding marker on the green curve. The green curve forms progress-sensitive local peaks at the five keyframes and rises sharply to 0.570.57 as the shoebox is closed, while the red curve stays in 0.04∼ 0.200.04\,\sim\,0.20 throughout and never crosses the green curve in any frames.

6.6 CRL Value Visualization

Before RealMan post-training, the contrastive heads score a held-out shoe-packing demonstration under its correct instruction and an incorrect bottle/cup instruction. The correct score rises overall from about 0.28 to 0.57, with increases near grasping, placement and lid closure; the incorrect score stays around 0.04–0.20. The curve is not strictly monotone. This single trajectory demonstrates instruction discrimination and progress sensitivity, but does not calibrate scores as probabilities or exhaustively distinguish physical feasibility from semantic mismatch.

6.7 Pre-training Efficiency

The fused role-aware CuTe kernel takes 0.531 ms per attention-layer forward pass versus 0.45 ms for BC-only FA3, a 1.18× ratio; the general FlexAttention alternatives are slower. Full-training throughput reaches about 477K tokens/s on 64 H100s, with reported scaling efficiencies of 95.2% at 32 GPUs and 85.1% at 64 relative to the two-GPU baseline. These measurements support modest kernel overhead and scalable implementation, not zero added cost. The microbenchmark prose says one H100 while Figure 10’s caption says two; the original discrepancy is retained.

Original figure
Figure 10: Pre-training efficiency of PRTS. (a) Per-layer attention forward time on 22 H100 GPUs for BC-only FA3, two FlexAttention backends, and the fused CuTe kernel. (b) Aggregate token throughput vs. number of H100 GPUs; dashed line: linear scaling from the 22-GPU point.

7 Conclusion

The clearest result is that adding a temporally structured contrastive objective during large-scale pretraining improves downstream policy robustness, especially in the matched perturbation ablation. The method remains dependent on labelled demonstrations, terminal timing assumptions, substantial pretraining and task-specific continuous-action fine-tuning. Open limits include low success on difficult instruction/layout shifts, uncalibrated value scores, small qualitative recovery studies and the absence of a proof covering arbitrary stochastic or failed trajectories.

Appendix A Contributions

The contribution statement attributes data collection and standardization to Jiangyuan Zhao, Yang Zhang and Fangzheng Yan; infrastructure to a broader engineering team; architecture/algorithm and project leadership to Yang Zhang; training and evaluation to Zhang, Zhao and Chenyou Fan; and advising to Chenjia Bai. The full fourteen-author byline is retained.

Appendix B Proof of Theorem 1

The proof minimizes the reverse contrastive cross-entropy by matching its predicted distribution to the normalized temporal targets. Taking logarithms yields a score linear in remaining time, plus a goal/batch normalization term. It then equates this time dependence with discounted occupancy for a deterministic expert trajectory reaching the goal at its terminal step, absorbing constant factors into the offset.

This is an idealized objective argument. The proof assumes the stated terminal-visit structure, treats the batch-dependent normalization in expectation, and does not establish that a finite normalized neural model attains the exact optimum. It supplies a rationale for temporal representation learning rather than a general probability-calibration theorem for deployed policies.

Equations 18–25
ℒl→sa(ϕ,ψ)=𝔼ℬ∼𝒟B[1|ℬ|∑i∈ℬ(−∑j∈𝒮⁡(li,ℬ)qi​jlogexp⁡(ψ​(li)⊤​ϕ​(sj,aj))∑k∈ℬexp⁡(ψ​(li)⊤​ϕ​(sk,ak)))],\displaystyle\mathcal{L}^{l\to\text{sa}}(\phi,\psi)=\mathbb{E}_{\mathcal{B}\sim\mathcal{D}^{B}}\left[\frac{1}{|\mathcal{B}|}\sum_{i\in\mathcal{B}}\left(-\sum_{j\in\mathcal{S}(l_{i};\mathcal{B})}q_{ij}\log\frac{\exp(\psi(l_{i})^{\top}\phi(s_{j},a_{j}))}{\sum_{k\in\mathcal{B}}\exp(\psi(l_{i})^{\top}\phi(s_{k},a_{k}))}\right)\right], (18)
Ll=−∑r=1mqrlogpr,\displaystyle\mathcal{L}_{l}=-\sum_{r=1}^{m}q_{r}\log p_{r}, (19)
ψ​(l)⊤​ϕ​(sjr,ajr)−log⁡Z=log⁡qr=(Tjr−tjr)​log⁡γ−log⁡(∑r′γTjr′−tjr′),\displaystyle\psi(l)^{\top}\phi(s_{j_{r}},a_{j_{r}})-\log Z=\log q_{r}=(T_{j_{r}}-t_{j_{r}})\log\gamma-\log\left(\sum_{r^{\prime}}\gamma^{T_{j_{r^{\prime}}}-t_{j_{r^{\prime}}}}\right), (20)
ψ​(l)⊤​ϕ​(sjr,ajr)=(Tjr−tjr)​log⁡γ+(logZ−log∑r′γTjr′−tjr′)⏟≜C⁡(l,ℬ).\displaystyle\psi(l)^{\top}\phi(s_{j_{r}},a_{j_{r}})=(T_{j_{r}}-t_{j_{r}})\log\gamma+\underbrace{\left(\log Z-\log\sum_{r^{\prime}}\gamma^{T_{j_{r^{\prime}}}-t_{j_{r^{\prime}}}}\right)}_{\triangleq C(l,\mathcal{B})}. (21)
Qlπ∗​(st,at)\displaystyle Q^{\pi^{*}}_{l}(s_{t},a_{t}) =(1−γ)∑k=0∞γk𝕀[st+k=sg]\displaystyle=(1-\gamma)\sum_{k=0}^{\infty}\gamma^{k}\mathbb{I}[s_{t+k}=s_{g}] (22)
=(1−γ)​γT−t,\displaystyle=(1-\gamma)\gamma^{T-t}, (23)
(T−t)​log⁡γ=log⁡Qlπ∗​(st,at)−log⁡(1−γ).\displaystyle(T-t)\log\gamma=\log Q^{\pi^{*}}_{l}(s_{t},a_{t})-\log(1-\gamma). (24)
ψ​(l)⊤​ϕ​(s,a)=log⁡Qlπ∗​(s,a)+(C⁡(l,ℬ)−log⁡(1−γ))⏟≜C⁡(l).\displaystyle\psi(l)^{\top}\phi(s,a)=\log Q^{\pi^{*}}_{l}(s,a)+\underbrace{\left(C(l,\mathcal{B})-\log(1-\gamma)\right)}_{\triangleq C(l)}. (25)

Comparison with Standard CRL.

Standard CRL samples future states geometrically; PRTS instead assigns explicit geometric weights to multiple state–action positives for a fixed language goal. This uses more same-task examples per batch and adapts the supervision to language labels. The claimed occupancy correspondence is conditional on the demonstration assumptions above, and the paper provides no separate quantitative variance comparison for the two estimators.

Appendix C Detailed real-world task descriptions and illustrations

The appendix retains all task illustrations, per-task physical success rates and controlled-generalization results. The descriptions specify what must be manipulated and how substeps compose, while the main protocol supplies retry and time limits.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 11: RealMan task suite. We evaluate PRTS on eleven RealMan tasks, including ten short-horizon desktop manipulation tasks and one long-horizon office task. The montage visualizes representative rollouts for the task settings described in Appendix C.

C.1 RealMan dual-arm tasks

The eleven dual-arm tasks cover paper disposal while holding the bin, block placement, serving a cup onto a coaster, transferring a tennis tube, handing over a pen holder, moving a cup and returning the arm, pressing left/right buttons, packing two shoes and closing the box, and alternating arms to stack four cups. The long office sequence places a mouse, inserts a pen, operates two switches and moves a book. Tables 8–9 retain all success cells and perturbation results; the task instructions make clear which operations are reused in the recombination tests.

Original PDF visual preserving complete panels, native vector labels and full embedded image detail.
Figure 12: Flexiv task suite. The Flexiv evaluation complements the RealMan suite with single-arm manipulation tasks that emphasize precise alignment, assembly, and insertion-like object placement.

C.2 Flexiv single-arm tasks

Flexiv performs three contact-sensitive sequences: aligning a four-hole plate with four posts, mounting three differently sized gears on designated posts, and placing two flowers into a glass receptacle. The original rollout panels show the objects and required placements. Each uses its own fine-tuned policy rather than the eleven-task shared RealMan checkpoint.

Task π0\pi_{0} π0.5\pi_{0.5} PRTS
Flip Tennis Tube 30.0 85.0 90.0
Hand Over 45.0 80.0 95.0
Office Long Term 5.0 40.0 95.0
Push Button Left 95.0 100.0 100.0
Push Button Right 100.0 100.0 100.0
Paper Rubbish 65.0 90.0 100.0
Place Block 90.0 100.0 100.0
Pick Shoes 90.0 90.0 95.0
Stack Cups 45.0 70.0 90.0
Serve Tea 90.0 90.0 95.0
Tea One 85.0 95.0 95.0
OVERALL 67.3 85.5 95.9
Table 8: Real-world evaluation on the RealMan dual-arm platform. Per-task success rates over 2020 real-world trials. PRTS achieves 95.9%95.9\% average SR and reaches at least 90%90\% on all 1111 tasks, outperforming π0.5\pi_{0.5} and π0\pi_{0} under the same evaluation protocol.
Method Paper Rubbish Place Block Pick Shoes Stack Cups
Light Pos Obj Task Light Pos Obj Task Light Pos Obj Task Light Pos Obj Task
π0\pi_{0} 60.0 50.0 60.0 20.0 45.0 40.0 45.0 5.0 80.0 80.0 70.0 0.0 35.0 10.0 0.0 30.0
π0.5\pi_{0.5} 90.0 85.0 95.0 35.0 100.0 100.0 100.0 15.0 40.0 70.0 85.0 40.0 45.0 25.0 5.0 50.0
PRTS (Ours) 100.0 100.0 100.0 80.0 100.0 100.0 100.0 55.0 95.0 95.0 95.0 85.0 90.0 90.0 80.0 75.0
Table 9: Controlled real-world generalization on the RealMan dual-arm platform. We report success rates under illumination (Light), spatial position (Pos), object instance (Obj), and task-instruction (Task) shifts for four representative RealMan tasks. The task-instruction axis is the most semantically demanding since the language goal changes rather than only the visual scene.

Appendix D Detailed results on LIBERO-Pro Task and Position variations

The full LIBERO-Pro breakdown shows where aggregate gains remain uneven. PRTS’s novel-instruction success ranges from 62.2% on Spatial to 8.8% on Object, with 33.8% Goal and 21.2% Long. Position results are 26.8%, 36.0%, 19.0% and 15.4%, respectively; π₀.₅ is better on Goal-Position at 38.0%. These cells qualify the aggregate claim of stronger generalization and reveal substantial remaining failures.

Method libero-spatial libero-object libero-goal libero-long
Pos (Avg.) Task (Avg.) Pos (Avg.) Task (Avg.) Pos (Avg.) Task (Avg.) Pos (Avg.) Task (Avg.)
OpenVLA [Kim et al., 2024] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
Qwen3-VL-PI [starVLA Contributors, 2025] (bs=32, 30K steps) 14.4 0.0 0.2 0.0 1.2 8.8 1.2 6.8
ABot-M0 [Yang et al., 2026] (bs=32, 30K steps) 13.4 54.8 9.0 9.8 5.6 10.6 0.2 14.0
π0\pi_{0} [Black et al., 2024] (bs=32, 30K steps) 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
π0.5\pi_{0.5} [Black et al., 2025] (bs=256, 30K steps) 20.0 1.0 17.0 1.0 38.0 0.0 8.0 1.0
PRTS (Ours) (bs=32, 30K steps) 26.8 62.2 36.0 8.8 19.0 33.8 15.4 21.2
Table 10: Detailed performance comparison on the Position and Task variations of LIBERO-Pro benchmark. Since VLAs are trained on a fixed instruction distribution, they perform poorly under task perturbations. While conventional VLAs suffer catastrophic performance drops under these distributional shifts, our PRTS model exhibits superior generalization capabilities over instruction following. The best results are highlighted in bold and second best results are underlined.
Concept demonstration

Open image ↗