Literature
Display
← All papers

Generalizing Motion Planners with Mixture of Experts for Autonomous Driving

Qiao Sun · Huimin Wang · Jiahao Zhan · Fan Nie · Xin Wen · Leimeng Xu · Kun Zhan · Peng Jia · Xianpeng Lang · Hang Zhao

· ICRA 2025 ↗

Robotics5 min text · CondensedOriginal paper ↗
Contents

At a glance

StateTransformer-2 is a raster-based driving planner with a vision encoder and a decoder-only mixture-of-experts backbone. Larger variants improve the reported NuPlan closed-loop scores, and a separate proprietary-data study finds decreasing trajectory-prediction loss with increasing data and model scale. The evidence comes from log-based evaluation and simulation; it does not demonstrate safe real-world operation or that expert routing directly discovers distinct driving rewards.

I INTRODUCTION

The paper studies whether simpler, scalable sequence models can generalize beyond common driving situations. It expands evaluation to larger NuPlan subsets, reactive-agent simulation, and synthetic InterPlan scenarios. STR2 predicts trajectories autoregressively, optionally using a discrete proposal token to reduce averaging across incompatible motion modes. The claims about balancing rewards are an interpretation of model behavior; training uses demonstrations rather than an explicitly identified reward model.

Refer to caption
Fig. 1: The planning results, in red, from PDM-Hybrid and STR2 at the pickup area at the top, and an illustration of the MoE model learning and balancing different explicit rewards at the bottom. In this case, STR2 produces a better-nudging trajectory by balancing two conflicting rewards, making progress, and avoiding collisions.

II Related Work

The comparison spans rule-based planning, imitation learning, and other learning-based planners. Simulation quality, dataset choice, and evaluation protocol can influence their relative performance; no approach has a general optimality or safety guarantee from these experiments.

II-A Imitation Learning based Planners

Imitation planners learn trajectories from demonstrations but can drift into unfamiliar states in closed-loop use. Prior systems add structured planning, augmentation, learned costs, or search. STR2 emphasizes model and data scaling, while still using trajectory sampling and scoring at deployment for the reported closed-loop setup.

II-B Scaling Laws

Scaling studies in sequence modeling motivate testing prediction loss against dataset and model size. Such trends are empirical relationships over the measured range, not guarantees that increased scale improves every driving metric.

Refer to caption
Fig. 2: An overview of the STR2-CPKS model which has a sequence of context, proposal, key points, and future states for the MoE backbone to model. For STR2-CKS, proposals are removed in the sequence for better efficiency. The context part has rasterized environment information encoded by scalable ViT encoders and past ego states.

II-C Simulation and Testings

NuPlan provides recorded urban driving and simulation-based evaluation. Open-loop trajectory similarity can diverge from closed-loop performance because of controller error and other agents’ responses. Reactive simulations remain simplified, so the paper uses several subsets and metrics rather than treating one score as a complete measure of generalization.

III Scaling for Generality with MoE

The planner uses two seconds of context to predict an eight-second ego trajectory, represented as a sequence of context, optional proposal, key points, and future states.

TABLE I: MoE Backbone Architecture
Parameter MoE-100m MoE-800m MoE-1b
dim 320 512 1024
n_layers 16 32 16
hidden_dim 1280 2048 4096
n_heads 16 32 16
n_kv_heads 8 8 8
num_experts 8 8 8
top_k_experts 2 2 2

III-A Training and Testing Datasets

Training removes scenario-type filters and uses more than seven million NuPlan scenarios. The reported larger evaluation sets contain 4,790 validation and 4,639 test scenarios. The accompanying description of sampling 100 scenarios for each of 14 types does not fully explain those totals. Val14, TestHard, reactive simulation, and InterPlan provide additional comparisons with different sources of difficulty.

TABLE II: Performance comparison with various state-of-the-arts with NuPlan Closed-Loop Metrics. NR is the overall score of the closed-loop simulation with non-reactive agents. R is the overall score of the closed-loop simulation with reactive agents controlled by a rule-based planner.
Val14 Set TestHard Set Val4k Set Test4k Set InterPlan
Methods NR ↑\uparrow R ↑\uparrow NR ↑\uparrow R ↑\uparrow NR ↑\uparrow R ↑\uparrow NR ↑\uparrow R ↑\uparrow Score ↑\uparrow
Expert (log replay) 94 80 85.96 68.80 93.08 76.91 93.12 75.01 14.76
IDM [28] 75.59 77.32 56.16 62.26 72.52 76.03 72.45 74.10 -
GameFormer [29] 82.95 83.88 66.59 68.83 - - - - -
PDM-Hybrid [6] 92.84 92.12 65.07 75.18 90.31 90.62 90.65 90.13 41.61
PlanTF [7] 84.72 76.25 72.59 60.62 80.52 70.84 80.90 70.62 30.53
DTPP [8] 71.66 66.09 60.11 63.66 68.43 65.24 71.98 70.20 30.32
STR-16m [26] 45.06 49.69 27.59 36.13 42.27 45.16 39.66 39.62 -
STR2-CPKS-100m (Ours) 92.79 92.18 74.25 78.40 90.45 91.14 90.66 90.25 42.13
STR2-CKS-800m (Ours) 92.32 92.12 74.21 78.58 90.64 91.15 90.35 90.37 44.77
STR2-CPKS-800m (Ours) 93.91 92.51 77.54 82.02 91.41 91.53 92.14 91.38 46.03
TABLE III: Performance comparison on testHard Set with details of the closed-loop reactive simulations. Higher scores indicate better performance. Col. refers to No at-fault Collisions. Drivable refers to Drivable area compliance. Direction refers to Driving direction compliance. Making Prog. refers to Ego progress along the expert’s route ratio. TTC refers to Time to Collision (TTC) within bound. Speed Limit refers to Speed limit compliance. Prog. on Route refers to Ego progress along the expert’s route ratio.
Methods Score Col. Drivable Direction Making Prog. TTC Speed Limit Prog. on Route Comfort
Expert (log replay) 68.80 77.02 95.96 98.16 100.00 69.85 94.12 98.48 99.26
GameFormer[29] 68.83 - - - - - - - -
PlanTF [7] 60.62 90.07 94.85 97.98 80.51 85.66 97.97 65.22 92.28
PDM-Hybrid [6] 75.18 95.22 95.58 99.08 93.38 84.19 99.53 75.47 83.45
Hoplan [30] 75.06 89.33 94.85 96.13 97.05 80.51 95.28 85.02 98.52
GUMP hybrid [21] 77.77 94.36 98.98 98.95 94.41 87.46 97.51 77.08 79.84
STR2-CPKS-100m (Ours) 78.40 96.51 96.32 98.90 94.49 85.29 99.70 77.91 83.46
STR2-CKS-800m (Ours) 78.58 96.32 96.69 98.90 94.49 84.56 99.70 79.29 86.02
STR2-CPKS-800m (Ours) 82.02 97.98 96.69 99.08 94.12 87.87 99.27 78.86 95.59

III-B Data Preprocess and Rasterization

Data preprocessing reduces precision to float32, merges pedestrian crowds into shapes, and removes unused road-user attributes. Fully stationary ten-second segments are filtered. Maps and agents are rasterized into 34 occupancy channels at two spatial scales, each 224 by 224, to represent both local detail and longer-range motion.

III-C StateTransformer-2

A 12-layer vision transformer encodes raster context; a causal transformer routes tokens through two of eight feed-forward experts per layer. Proposal-equipped variants classify among 512 clustered, dynamically feasible trajectories before generating continuous outputs. For closed-loop evaluation, the system combines predicted paths, lateral offsets, and target-speed choices into 21 candidates, rolls them through the controller, and scores the resulting trajectories. Thus the final planner includes structured search and scoring in addition to the learned model.

TABLE IV: Performance comparison with various state-of-the-arts with NuPlan Open-Loop Metrics.
Val4k Set Test4k Set Val14 Set
Methods OLS ↑\uparrow 8sADE ↓\downarrow 8sFDE ↓\downarrow OLS ↑\uparrow 8sADE ↓\downarrow 8sFDE ↓\downarrow OLS ↑\uparrow 8sADE ↓\downarrow 8sFDE ↓\downarrow
PlanCNN [32] - - - - - - 64 2.468 5.936
PDM-Hybrid [6] 84.06 2.435 5.202 82.01 2.618 5.546 84 2.382 5.068
PlanTF [7] 88.59 1.774 3.892 87.30 1.855 4.042 89.18 1.697 3.714
DTPP [8] 65.15 4.196 9.231 64.18 4.117 9.181 67.33 4.088 8.846
STR-124m [26] 81.88 1.939 4.968 82.68 2.003 4.972 88.0 1.777 4.515
STR2-CKS-800m (Ours) 90.07 1.473 4.124 89.12 1.537 4.269 89.2 1.496 4.210

III-D Training and Testing Settings

The encoder and backbone train from scratch for 20 epochs. The reported 800-million-parameter run uses eight 96-GB H20 accelerators; the 100-million run uses eight RTX 3090 GPUs. Batched simulation inference speeds evaluation, with each larger test set taking about five hours. The learning-rate schedule restarts each epoch and training uses bfloat16.

III-E Scaling on the LiAuto Dataset

The separate LiAuto study uses proprietary urban-driving logs with navigation maps and tracked road users. After filtering route mismatches, ten-second windows provide over one billion training samples. Windows are not equivalent to independent driving events, and the paper does not provide open access to this dataset. Mention of a large vehicle fleet describes the data and deployment context, not a reported controlled deployment trial of STR2.

TABLE V: Scaling study on the impact of model parameters with STR2-CKS evaluated on NuPlan Open-Loop Metrics.
STR Val14-OLS Val14-8sADE Val14-8sFDE MR
STR2-100m 88.56 1.55 4.47 3.05%
STR2-800m 89.02 1.48 4.20 2.87%
STR2-1b 89.74 1.46 4.13 2.15%
Refer to caption
Fig. 3: Scaling results with the size of the training dataset, counted as the number of tokens DD in the left and scaling results with model parameters NN in the right. All axes are logarithmically scaled.

IV Results

The results distinguish open-loop trajectory accuracy, closed-loop simulator scores, and proprietary-data scaling. Improvements in one category should not be assumed to establish gains in the others.

IV-A Open-Loop Performance

The reported 800-million STR2 model improves eight-second average displacement error over the listed baselines, but PlanTF has lower final displacement error on all three tabled subsets. The separate scaling table shows Val14 average error decreasing from 1.55 to 1.48 to 1.46 as model size rises from 100 million to 800 million to one billion parameters. These are prediction results, not direct real-world safety measures.

IV-B Closed-Loop Performance

The strongest proposal-equipped 800-million model leads the reported aggregate closed-loop comparisons, reaching 82.02 on reactive TestHard and 46.03 on InterPlan. Individual component metrics do not all improve uniformly. Reactive-agent behavior is rule-based and limited; score drops in some baselines suggest sensitivity but do not isolate causality confusion as the only explanation. GameFormer lacks a released checkpoint for the authors’ re-evaluation, so its available numbers are not uniformly reproduced across the new sets.

IV-C Scaling to Billions on LiAuto Dataset

On the LiAuto data, measured prediction loss declines approximately as a power law across the tested data and model scales. This analysis reaches one billion samples and roughly 300 million model parameters, distinct from the larger NuPlan model sweep. It evaluates trajectory L2 loss rather than a matched sweep of closed-loop driving outcomes; extrapolation beyond the fitted range remains untested.

V CONCLUSIONS

Scaling a mixture-of-experts sequence planner improves several reported benchmarks, but limitations remain in simulator interaction fidelity, proprietary-data reproducibility, and inference latency on edge hardware. Larger-data and larger-model studies, stronger interactive simulators, and faster deployment are proposed future work. The results do not establish that model scale alone resolves autonomous-driving generalization.

ACKNOWLEDGMENT

The authors acknowledge contributions to simulator acceleration, baseline evaluation, training, sampling, and simulation analysis.

Concept demonstration

Open image ↗