Literature
Display
← All papers

Anchor3R: Streaming 3D Reconstruction with Transient Anchors for Long-Horizon Visual Mapping

Peilin Tao · Chong Cheng · Yuansen Du · Caiwei Song · Zhengqing Chen · Xiaoyang Guo · Wei Yin · Weiqiang Ren · Qian Zhang · Hainan Cui · Shuhan Shen

Preprint posted · CoRL 2026 (accepted) ↗

Robotics9 min text · CondensedOriginal paper ↗
Contents

Overview

Code. This edition follows the complete nineteen-page, 14 September 2026 v2, including six numbered figures, seven tables, seventeen numbered equations, two algorithms and all 57 references.

Complete original PDF block S0.F1
Figure 1: Anchor3R results on campus-scale sequences. We visualize our reconstruction on campus_train0 and campus_train1 of the VBR dataset, two real-world sequences with 12,042 and 11,671 frames. Predicted trajectories are color-coded and ground truth is shown as white curves.

At a glance

Anchor3R predicts recent cameras relative to each incoming image, then combines these local measurements in a pose graph. Its transformer caches image evidence while rebuilding pose queries for every window. Online inference has bounded accelerator memory; optional offline loop retrieval and motion averaging further reduce drift. Experiments replay real datasets, with no physical-robot deployment and no constant bound on total map storage.

1 Introduction

Maintaining a distant coordinate origin inside a recurrent reconstruction model can couple local geometry with accumulated drift. Anchor3R instead uses each current image as a transient reference, leaving global alignment to an explicit graph. Overlapping windows contribute redundant relative-pose constraints, making later refinement possible.

The distinction matters operationally: strict online prediction has no loop detection and only estimates the new pose, whereas offline refinement revisits the graph and adds long-range constraints. The latter produces the strongest long-trajectory results.

2 Related Work

Classical SfM and SLAM.

Incremental structure from motion uses correspondence geometry, camera registration, triangulation and bundle adjustment. Global approaches and motion averaging separate relative measurements from global recovery. Anchor3R keeps this decomposition while learning the relative measurements.

Offline Feed-forward Reconstruction.

DUSt3R, VGGT and π³ provide feed-forward geometric priors. Chunked or stateful methods such as VGGT-Long, ZipMap and LoGeR extend scale, chiefly through offline/chunk processing. Anchor3R makes local measurements as frames arrive and exposes them for graph refinement.

Streaming Feed-forward Reconstruction.

CUT3R, STream3R, WinT3R and LongStream use persistent state, causal caches or compact camera pools. Anchor3R avoids carrying explicit historical pose-query states between local reference frames; its image cache and motion graph serve different purposes.

3 Method

A bounded window predicts relative poses plus a pointmap in the newest camera frame. Online medians combine those predictions with recovered historical poses. Offline robust rotation/position averaging uses the accumulated graph and optional loop edges to redistribute drift.

3.1 Current-Centric Formulation

The active window contains at most W recent frames. Equation 1 maps the current camera into each source camera; the current-to-current transform is identity. Each earlier frame supplies an edge to the current anchor. Overlap creates redundant short-range constraints, rather than only one consecutive-frame chain.

“Dense” describes this local measurement graph, not every pair in the entire recording. With fixed W, edge storage grows approximately linearly with sequence length. Local pointmaps are aligned by recovered global poses.

Equation 1
𝐓i←t=𝐓i​𝐓t−1,𝐓t←t=𝐈.\mathbf{T}_{i\leftarrow t}=\mathbf{T}_{i}\mathbf{T}_{t}^{-1},\qquad\mathbf{T}_{t\leftarrow t}=\mathbf{I}. (1)
Complete original PDF block S3.F2
Figure 2: Overview of Anchor3R. Given the current frame ItI_{t}, Anchor3R extracts DINOv2 patch tokens and instantiates grouped pose-query tokens for frames in 𝒲t\mathcal{W}_{t}, using ItI_{t} as the local reference. A sliding-window pose-query Transformer alternates between decoupled frame attention and current-centric window attention. The camera head decodes window-relative poses, while the point head predicts the current-frame pointmap.

3.2 Sliding-Window Pose-Query Transformer

DINOv2 supplies patch features. Every active frame gets one pose token plus 31 registers, with different learned templates for current-reference and source frames. Decoupled frame attention computes image features; pose queries read their corresponding image states. Window attention combines the current patches and all active pose-query groups.

Only image-side keys/values are cached; pose-query states are discarded and recreated. This avoids explicitly reusing hypotheses tied to a previous reference. The equations below retain the grouping and attention operations; the empirical ablation supports this cache design without establishing a formal independence guarantee.

Equations
𝐐t0=Concati∈𝒲t⁡𝐐t,i0\mathbf{Q}_{t}^{0}=\operatorname{Concat}_{i\in\mathcal{W}_{t}}\mathbf{Q}_{t,i}^{0}
(𝐏~tl,𝐊~tl,𝐕~tl)=ℱl​(𝐏tl−1)(\tilde{\mathbf{P}}_{t}^{\,l},\tilde{\mathbf{K}}_{t}^{\,l},\tilde{\mathbf{V}}_{t}^{\,l})=\mathcal{F}^{\,l}(\mathbf{P}_{t}^{\,l-1})
𝐐~t,il=ℱl​(𝐐t,il−1,𝐊~il,𝐕~il)\tilde{\mathbf{Q}}_{t,i}^{\,l}=\mathcal{F}^{\,l}(\mathbf{Q}_{t,i}^{\,l-1};\tilde{\mathbf{K}}_{i}^{\,l},\tilde{\mathbf{V}}_{i}^{\,l})
(𝐏tl,{𝐐t,il}i∈𝒲t,𝐊¯tl,𝐕¯tl)=𝒲l​(Concat⁡(𝐏~tl,{𝐐~t,il}i∈𝒲t),𝐊¯𝒲tl,𝐕¯𝒲tl)(\mathbf{P}_{t}^{\,l},\{\mathbf{Q}_{t,i}^{\,l}\}_{i\in\mathcal{W}_{t}},\bar{\mathbf{K}}_{t}^{\,l},\bar{\mathbf{V}}_{t}^{\,l})=\mathcal{W}^{\,l}(\operatorname{Concat}(\tilde{\mathbf{P}}_{t}^{\,l},\{\tilde{\mathbf{Q}}_{t,i}^{\,l}\}_{i\in\mathcal{W}_{t}});\bar{\mathbf{K}}_{\mathcal{W}_{t}}^{\,l},\bar{\mathbf{V}}_{\mathcal{W}_{t}}^{\,l})

3.3 Prediction Heads

The camera head jointly decodes the first token of each query group because the outputs share one current-frame reference. A DPT-style head combines multiple patch-feature levels to output three local coordinates and one confidence value per pixel. Historical pointmaps are not re-predicted after later observations.

Equations
{𝐓^i←t}i∈𝒲t=ℋcam​(Concati∈𝒲t⁡𝐐t,iL​[1])\{\hat{\mathbf{T}}_{i\leftarrow t}\}_{i\in\mathcal{W}_{t}}=\mathcal{H}_{\mathrm{cam}}(\operatorname{Concat}_{i\in\mathcal{W}_{t}}\mathbf{Q}_{t,i}^{\,L}[1])
[𝐗^tlocal,𝚺^t]=ℋpt​({𝐏tl}l∈ℒ)∈ℝH0×W0×4\big[\hat{\mathbf{X}}_{t}^{\mathrm{local}},\hat{\mathbf{\Sigma}}_{t}\big]=\mathcal{H}_{\mathrm{pt}}(\{\mathbf{P}_{t}^{\,l}\}_{l\in\mathcal{L}})\in\mathbb{R}^{H_{0}\times W_{0}\times 4}

3.4 Motion Graph and Pose Recovery

World-to-camera rotations define relative rotation RᵢRₜᵀ. Rotating a predicted relative translation back by Rᵢᵀ gives a displacement from historical center cᵢ to cₜ. Online inference aggregates candidate current rotations through iterative Lie-algebra medians and candidate centers through coordinate-wise medians.

Offline refinement fixes a gauge, applies robust rotation averaging, then solves an L1 center problem with ADMM. Retrieved old views can be reinserted to predict loop edges. This can redistribute errors, but does not guarantee a globally accurate reconstruction or repair inconsistent learned scales.

Equation 2
min⁡∑(i,t)∈ℰ{𝐑i}⁡ρR​(‖Log⁡(𝐑^i←t⊤​𝐑i​𝐑t⊤)‖2),min⁡∑(i,t)∈ℰ{𝐜i}⁡‖𝐜i−𝐜t+𝒗^i,t‖1.\min_{\{\mathbf{R}_{i}\}}\sum_{(i,t)\in\mathcal{E}}\rho_{R}\!\left(\left\|\operatorname{Log}\!\big(\hat{\mathbf{R}}_{i\leftarrow t}^{\top}\mathbf{R}_{i}\mathbf{R}_{t}^{\top}\big)\right\|_{2}\right),\qquad\min_{\{\mathbf{c}_{i}\}}\sum_{(i,t)\in\mathcal{E}}\left\|\mathbf{c}_{i}-\mathbf{c}_{t}+\hat{\boldsymbol{v}}_{i,t}\right\|_{1}. (2)

3.5 Training

Training objectives.

The objective combines relative-camera supervision with confidence-weighted point reconstruction and spatial-gradient matching. A shared L1-fitted scale multiplies both predicted translations and pointmaps; a negative log-confidence term discourages trivially suppressing residuals. The exact four loss/scale expressions are retained below. This encourages consistency without establishing absolute metric scale.

Equations
ℒ=λcam​ℒcam+ℒpmap\mathcal{L}=\lambda_{\mathrm{cam}}\mathcal{L}_{\mathrm{cam}}+\mathcal{L}_{\mathrm{pmap}}
ℒcam=∑t∑i∈𝒲t(‖qi←t−q^i←t‖1+‖𝐭i←t−s∗​𝐭^i←t‖1)\mathcal{L}_{\mathrm{cam}}=\sum_{t}\sum_{i\in\mathcal{W}_{t}}\big(\|q_{i\leftarrow t}-\hat{q}_{i\leftarrow t}\|_{1}+\|\mathbf{t}_{i\leftarrow t}-s^{*}\hat{\mathbf{t}}_{i\leftarrow t}\|_{1}\big)
ℒpmap=∑t‖𝚺^t⊙(𝐗tlocal−s∗​𝐗^tlocal)‖+‖𝚺^t⊙(∇𝐗tlocal−∇(s∗​𝐗^tlocal))‖−α​log⁡𝚺^t\mathcal{L}_{\mathrm{pmap}}=\sum_{t}\|\hat{\mathbf{\Sigma}}_{t}\odot(\mathbf{X}_{t}^{\mathrm{local}}-s^{*}\hat{\mathbf{X}}_{t}^{\mathrm{local}})\|+\|\hat{\mathbf{\Sigma}}_{t}\odot(\nabla\mathbf{X}_{t}^{\mathrm{local}}-\nabla(s^{*}\hat{\mathbf{X}}_{t}^{\mathrm{local}}))\|-\alpha\log\hat{\mathbf{\Sigma}}_{t}
s∗=arg​mins∑t∥𝐗tlocal−s𝐗^tlocal∥1s^{*}=\operatorname*{arg\,min}_{s}\sum_{t}\|\mathbf{X}_{t}^{\mathrm{local}}-s\hat{\mathbf{X}}_{t}^{\mathrm{local}}\|_{1}

Training data.

Training mixes sixteen real/synthetic sources: WildRGB, ScanNet, HyperSim, Mapillary, Replica, Mapfree, TartanAir, MVS-Synth, Virtual KITTI, Aria Synthetic Environments, Spring, Waymo, BlendedMVS, Co3Dv2, MegaDepth and DL3DV. Pose-guided sampling makes overlapping sequences from unordered data; random temporal spacing and block shuffling augment videos.

Implementation details.

VGGT initializes a 24-layer, approximately 1.2-billion-parameter model. Training unrolls 48 frames with W=10, one pose token plus 31 registers per frame, and images at most 518 pixels on their longer side. AdamW uses peak rate 10⁻⁴, cosine decay and 2,000 warm-up steps. Eighty thousand iterations on 32 A800 GPUs take about twelve days.

4 Experiments

Evaluation covers full replayed driving, indoor and mobile-mapping sequences. Main sequences are excluded from training; Waymo evaluation uses nine held-out segments from its official training split. Small-model diagnostic ablations use Virtual KITTI, which is also among training sources.

Complete original PDF block S4.T1
Table 1: Quantitative comparison on KITTI [15] in terms of ATE. The upper block lists optimization-based baselines, and the lower block reports streaming feed-forward methods. Failed cases are marked by ∗* and excluded from the average. Anchor3R achieves the best average accuracy.
Original selectable table
Methods KITTI [15] (ATE ↓\downarrow) Avg.
00 01 02 03 04 05 06 07 08 09 10
4541×\times, 3.7 km 1101×\times, 2.5 km 4661×\times, 5.1 km 801×\times, 0.6 km 271×\times, 0.4 km 2761×\times, 2.2 km 1101×\times, 1.2 km 1101×\times, 0.7 km 4071×\times, 3.2 km 1591×\times, 1.7 km 1201×\times, 0.9 km
FastVGGT * 705.39 * 62.38 10.27 157.74 124.43 69.27 * 190.10 194.75 189.29
MASt3R-SLAM * 530.37 * 18.87 88.98 159.430 92.00 * * * * 177.93
VGGT-SLAM * 607.16 * 169.83 13.12 * * * * * * 263.37
VGGT-Long 8.64 21.20 52.72 8.78 4.20 9.88 4.67 2.66 72.98 31.84 27.71 25.94
CUT3R 185.89 651.52 296.98 148.06 22.17 155.61 132.54 77.03 238.39 205.94 193.39 209.78
TTT3R 190.93 546.84 218.77 105.28 11.62 153.12 132.94 70.95 180.57 211.01 133.00 177.73
STream3R 190.98 681.95 301.40 158.25 102.73 159.85 135.03 90.37 261.15 216.31 207.49 227.77
StreamVGGT 191.93 653.06 303.35 157.50 108.24 160.46 133.71 89.00 263.95 216.69 209.80 226.15
LongStream 92.55 46.01 134.70 3.81 1.95 84.69 23.12 14.93 62.07 85.61 21.48 51.90
Anchor3R-Online 44.18 48.43 149.61 4.39 1.99 62.63 12.12 10.39 52.12 55.23 8.57 40.89
Anchor3R-Offline 19.68 48.62 75.75 4.22 1.90 7.56 6.36 7.63 49.34 49.02 8.00 25.03
Complete original PDF block S4.T2
Table 2: Quantitative comparison on VBR [5]. We report ATE, where lower is better.
Original selectable table
Method VBR [5] ATE ↓\downarrow Avg.
campus_train0 campus_train1 ciampino_train1 colosseo_train0 diag_train0 pincio_train0 spagna_train0
12042×\times, 2.73 km 11671×\times, 2.95 km 18846×\times, 5.20 km 8815×\times, 1.45 km 10021×\times, 1.02 km 11142×\times, 1.27 km 14141×\times, 1.56 km
VGGT-SLAM 93.51 71.74 124.10 101.00 33.64 66.42 57.00 78.20
VGGT-Long 118.59 98.21 172.13 39.56 30.80 53.44 50.27 80.43
Pi3-Chunk 78.50 65.77 111.72 77.09 23.81 41.99 44.76 63.38
InfiniteVGGT 123.65 100.00 * 83.91 31.58 70.73 56.25 91.60
LongStream 100.57 105.55 131.78 72.52 32.35 43.47 59.31 77.93
Anchor3R-Online 86.63 82.16 168.25 61.43 29.38 51.34 54.34 76.21
Anchor3R-Offline 5.13 3.52 78.64 17.25 5.35 15.56 11.76 19.60

4.1 Camera Pose Estimation

Evaluation protocol.

Trajectories receive a similarity alignment before absolute trajectory error is measured. This removes global scale as well as rigid gauge error; it does not assess absolute metric-scale accuracy. Failed runs are excluded from averages, so methods with failures are not averaged over the same successful-sequence set.

Online mode processes frames once in temporal order, without loop retrieval. Offline mode selects every fifth frame as a keyframe, retrieves three similar views with NetVLAD, reinserts temporally separated matches and averages the augmented motion graph.

Results.

On KITTI, average aligned error falls from LongStream 51.90 m to Anchor3R-Online 40.89 m and Offline 25.03 m. Offline wins the main-table average, not every route. On VBR, Online 76.21 m only slightly improves over LongStream 77.93 m; Offline 19.60 m is best across the main table’s seven routes.

Table 3 reports Online 0.091 m on TUM, 17.661 m on Oxford Spires and 0.425 m on Waymo. Oxford/Waymo are best in that table; TUM trails LongStream 0.076 and MASt3R-SLAM 0.082. Appendix D’s matched-backend comparison further qualifies the offline ranking.

Complete original PDF block S4.T3
Table 3: Quantitative comparison on TUM, Oxford Spires, and Waymo. Top: optimization-based methods; Bottom: streaming methods. Anchor3R remains robust across short indoor trajectories and mobile mapping sequences.
Original selectable table
Methods TUM [41] Oxford Spires [43] Waymo [42]
ATE ↓\downarrow ATE ↓\downarrow ATE ↓\downarrow
FastVGGT 0.418 36.577 1.281
MASt3R-SLAM 0.082 37.728 7.625
VGGT-SLAM 0.123 31.003 7.431
CUT3R 0.542 32.440 9.396
TTT3R 0.308 36.214 3.486
STream3R 0.633 37.569 42.203
StreamVGGT 0.627 37.255 45.101
LongStream 0.076 19.815 0.737
Anchor3R-Online 0.091 17.661 0.425
Complete original PDF block S4.T4
Table 4: Quantitative comparison on 7Scenes and TUM. CD ↓\downarrow and F1@0.25 ↑\uparrow are adopted for evaluation. Best numbers are in bold; second-best numbers are underlined.
Original selectable table
Methods 7Scenes TUM
CD ↓\downarrow F1@0.25 ↑\uparrow CD ↓\downarrow F1@0.25 ↑\uparrow
FastVGGT 6.373 0.710 0.104 0.926
MASt3R-SLAM 5.987 0.691 0.057 0.954
VGGT-SLAM 6.306 0.696 1.993 0.633
CUT3R 6.281 0.274 0.474 0.533
TTT3R 6.231 0.260 0.249 0.792
STream3R 6.353 0.479 1.126 0.444
StreamVGGT 6.630 0.483 0.680 0.402
LongStream 2.260 0.641 0.225 0.673
Anchor3R-Online 1.848 0.707 0.108 0.933

4.2 3D Reconstruction

Evaluation protocol.

Predicted poses and pointmaps form a cloud, which is similarity-aligned to ground truth. Evaluation uses Chamfer distance and F1 at threshold 0.25 on 7Scenes and TUM. These aligned geometry metrics do not validate unaligned global scale.

Results.

On 7Scenes, Online achieves Chamfer 1.848 and F1=0.707: best distance, but slightly below FastVGGT’s F1=0.710. On TUM, 0.108/0.933 trails MASt3R-SLAM 0.057/0.954; FastVGGT also has slightly smaller distance 0.104. The method improves streaming map consistency without universally beating optimization-based systems.

4.3 Ablation and Analysis

Most diagnostics use a ViT-Small model on Virtual KITTI. They test design choices but are not identical in scale or distribution to the main 1.2B model.

Current-centric prediction.

Sixty-frame attention maps contrast persistent first-frame attention with support moving along the current window. This illustrates the intended local-reference behavior; attention visualization alone does not establish the cause of all accuracy gains.

Image-only key/value cache.

Removing cached pose tokens lowers mean ATE 3.517→2.642, relative translation error 0.146→0.097 and relative rotation error 0.092→0.090. Most cells improve, with exceptions visible in Table 5. Reinstantiating queries helps empirically; image features still participate in window attention.

Complete original figure S4.F3
Figure 3: Attention-score comparison. The first-centric baseline exhibits persistent attention to the first frame throughout the sequence, while Anchor3R shifts its attention support forward with the current-centric sliding window.
Complete original figure S4.F4
Figure 4: Qualitative comparison on KITTI. Loop-closure constraints make the offline trajectories of sequences 00 and 05 more globally consistent and closer to the ground truth.
Complete original PDF block S4.T5
Table 5: Quantitative comparison on vKITTI. For each scene, we report ATE, relative rotation error (RRE), and relative translation error (RTE). The number of images and trajectory length are shown under each scene name. Best results are highlighted in bold.
Original selectable table
Methods Scene 01 Scene 02 Scene 06 Scene 18 Scene 20 Avg.
447×, 332m447\times,\,332\,\mathrm{m} 223×, 113m223\times,\,113\,\mathrm{m} 270×, 51m270\times,\,51\,\mathrm{m} 339×, 254m339\times,\,254\,\mathrm{m} 837×, 711m837\times,\,711\,\mathrm{m}
ATE ↓\downarrow RTE ↓\downarrow RRE ↓\downarrow ATE ↓\downarrow RTE ↓\downarrow RRE ↓\downarrow ATE ↓\downarrow RTE ↓\downarrow RRE ↓\downarrow ATE ↓\downarrow RTE ↓\downarrow RRE ↓\downarrow ATE ↓\downarrow RTE ↓\downarrow RRE ↓\downarrow ATE ↓\downarrow RTE ↓\downarrow RRE ↓\downarrow
w pose cache 3.613 0.177 0.132 2.174 0.111 0.074 0.285 0.037 0.055 3.955 0.195 0.087 7.558 0.211 0.112 3.517 0.146 0.092
w/o pose cache 3.113 0.101 0.127 1.054 0.084 0.073 0.424 0.042 0.059 1.410 0.118 0.080 7.208 0.142 0.110 2.642 0.097 0.090

Active window size.

Increasing the small-model window 5→10 improves ATE 3.23→2.64, rotation 0.12→0.09 and translation 0.15→0.10. Window 15 gives 2.55/0.08/0.10 but reduces speed 27.5→22.9 FPS and raises reported iteration time 2.7→3.2 s. W=10 is the selected trade-off.

Complete original PDF block S4.F5
Figure 5: Runtime and GPU memory scaling under the same resolution and hardware. Anchor3R, trained on 48-frame sequences, maintains nearly constant GPU memory and approximately linear runtime up to 10K frames.

Runtime and memory scaling.

The full model runs at 12.7 FPS with 11.9 GB peak accelerator memory; measured inference time grows approximately linearly through 10,000 frames. Memory boundedness applies to online accelerator state. Graphs and pointmaps on processor memory/disk grow with the recording, and offline global solving is an additional stage.

Reported motion-averaging times are 11.88 s for 12K and 17.24 s for 18.8K frames. These are solver timings, not an end-to-end bound for retrieval, anchor reinsertion, prediction and refinement. Training uses 48-frame clips; long-sequence inference is tested, not guaranteed for arbitrary duration.

Controlled offline refinement.

Applying the same retrieval/refinement protocol helps other models too. Anchor3R wins all seven routes against LongStream-Off-5 and six against VGGT-Off-5. However, VGGT-Off-5 has the better average: 16.96 m versus 19.60 m, because its ciampino error 28.30 m is much below Anchor3R’s 78.64 m. Route-win count and aggregate error tell different stories.

5 Limitations

Pointmaps are predicted once, so pose refinement cannot update their local geometry. Relative-scale inconsistency can degrade averaging. Highly dynamic scenes lack dedicated evaluation, loop retrieval/alignment remain separate from the learned model, and all experiments are dataset replay without physical-robot deployment.

6 Conclusion

Current-frame-relative prediction, transient pose queries and graph recovery make long streams practical and correctable. The evidence supports bounded online accelerator memory and improved aligned trajectories/maps, with substantial gains from offline loop constraints and remaining scale, dynamics and difficult-turn failures.

Acknowledgments

Support is acknowledged from the National Natural Science Foundation of China and Beijing Natural Science Foundation.

Supplementary Material

Appendix A Online Motion Averaging

Each historical pose and relative rotation induces a candidate current rotation. Algorithm 1 runs several randomly initialized hypotheses, computes coordinate-wise medians of logarithmic rotation residuals, and left-multiplies the corresponding exponential update. It selects the converged hypothesis with smallest median angular residual; if none converges, it returns the first candidate.

Candidate camera centers are aggregated by coordinate-wise median without iteration. Small-angle handling and multiple initializations improve numerical robustness, but the median procedure has no stated global-optimality guarantee. Previously recovered poses remain fixed in this online update.

Equations 3–8
𝐑^i←t≈𝐑i​𝐑t⊤.\hat{\mathbf{R}}_{i\leftarrow t}\approx\mathbf{R}_{i}\mathbf{R}_{t}^{\top}. (3)
𝐑~t(i)=𝐑^i←t⊤​𝐑^i.\tilde{\mathbf{R}}_{t}^{(i)}=\hat{\mathbf{R}}_{i\leftarrow t}^{\top}\hat{\mathbf{R}}_{i}. (4)
𝐫i(m)=Log⁡(𝐑~t(i)​(𝐑(m))⊤)∈ℝ3.\mathbf{r}_{i}^{(m)}=\operatorname{Log}\!\left(\tilde{\mathbf{R}}_{t}^{(i)}(\mathbf{R}^{(m)})^{\top}\right)\in\mathbb{R}^{3}. (5)
Δ​𝝎(m)=mediani⁡{𝐫i(m)},𝐑(m)←Exp⁡(Δ​𝝎(m))​𝐑(m).\Delta\boldsymbol{\omega}^{(m)}=\operatorname{median}_{i}\{\mathbf{r}_{i}^{(m)}\},\qquad\mathbf{R}^{(m)}\leftarrow\operatorname{Exp}(\Delta\boldsymbol{\omega}^{(m)})\mathbf{R}^{(m)}. (6)
𝐜~t(i)=𝐜^i+𝐯^i,t,𝐯^i,t≈𝐜t−𝐜i.\tilde{\mathbf{c}}_{t}^{(i)}=\hat{\mathbf{c}}_{i}+\hat{\mathbf{v}}_{i,t},\qquad\hat{\mathbf{v}}_{i,t}\approx\mathbf{c}_{t}-\mathbf{c}_{i}. (7)
𝐜^t=mediani∈𝒲t∖{t}⁡{𝐜~t(i)}.\hat{\mathbf{c}}_{t}=\operatorname{median}_{i\in\mathcal{W}_{t}\setminus\{t\}}\{\tilde{\mathbf{c}}_{t}^{(i)}\}. (8)
Complete original PDF block alg1
Algorithm 1 Online Lie-Algebra Median Rotation Averaging
Original selectable algorithm
1: Candidate rotations {𝐑~j}j=1M\{\tilde{\mathbf{R}}_{j}\}_{j=1}^{M}, number of initializations NinitN_{\mathrm{init}}, maximum iterations KK, threshold ϵ\epsilon
2: Robust current rotation 𝐑^t\hat{\mathbf{R}}_{t}
3: Randomly initialize {𝐑(m)}m=1Ninit\{\mathbf{R}^{(m)}\}_{m=1}^{N_{\mathrm{init}}} from {𝐑~j}j=1M\{\tilde{\mathbf{R}}_{j}\}_{j=1}^{M}
4: Set scores e(m)←+∞e^{(m)}\leftarrow+\infty and convergence flags c(m)←falsec^{(m)}\leftarrow\mathrm{false}
5: for k=1k=1 to KK do
6:   for m=1m=1 to NinitN_{\mathrm{init}} do
7:    if c(m)=truec^{(m)}=\mathrm{true} then
8:      continue
9:    end if
10:    for j=1j=1 to MM do
11:      𝐫j(m)←Log⁡(𝐑~j​(𝐑(m))⊤)\mathbf{r}_{j}^{(m)}\leftarrow\operatorname{Log}\!\left(\tilde{\mathbf{R}}_{j}(\mathbf{R}^{(m)})^{\top}\right)
12:      θj(m)←‖𝐫j(m)‖2\theta_{j}^{(m)}\leftarrow\|\mathbf{r}_{j}^{(m)}\|_{2}
13:    end for
14:    Δ​𝝎(m)←medianj⁡{𝐫j(m)}\Delta\boldsymbol{\omega}^{(m)}\leftarrow\operatorname{median}_{j}\{\mathbf{r}_{j}^{(m)}\}
15:    if ‖Δ​𝝎(m)‖2<ϵ\|\Delta\boldsymbol{\omega}^{(m)}\|_{2}<\epsilon then
16:      e(m)←medianj⁡{θj(m)}e^{(m)}\leftarrow\operatorname{median}_{j}\{\theta_{j}^{(m)}\}
17:      c(m)←truec^{(m)}\leftarrow\mathrm{true}
18:    else
19:      𝐑(m)←Exp⁡(Δ​𝝎(m))​𝐑(m)\mathbf{R}^{(m)}\leftarrow\operatorname{Exp}(\Delta\boldsymbol{\omega}^{(m)})\mathbf{R}^{(m)}
20:    end if
21:   end for
22:   if all hypotheses have converged then
23:    break
24:   end if
25: end for
26: if at least one hypothesis converged then
27:   m⋆←arg⁡minm⁡e(m)m^{\star}\leftarrow\arg\min_{m}e^{(m)}
28:   𝐑^t←𝐑(m⋆)\hat{\mathbf{R}}_{t}\leftarrow\mathbf{R}^{(m^{\star})}
29: else
30:   𝐑^t←𝐑~1\hat{\mathbf{R}}_{t}\leftarrow\tilde{\mathbf{R}}_{1}
31: end if
32: return 𝐑^t\hat{\mathbf{R}}_{t}

Appendix B Offline Motion Averaging

Offline recovery jointly uses local-window edges and any reinserted loop edges. Online estimates initialize the rotations; one reference frame fixes the gauge. This global pass is distinct from the causal online update.

Rotation averaging.

Equations 9–14 and Algorithm 2 retain the exact rotation procedure. Right-trivialized residuals are linearized into a sparse incidence system; Geman–McClure-style weights use a five-degree scale. Weighted normal equations produce axis-angle increments, applied by right multiplication until the mean increment is small or the iteration limit is reached.

These are robust local iterative updates, not a proof that all corrupted loop or local measurements will be rejected. The arbitrary reference gauge is removed again by evaluation alignment.

Equations 9–14
𝐑i(0)=𝐑^ionline​(𝐑^W−1online)⊤.\mathbf{R}_{i}^{(0)}=\hat{\mathbf{R}}^{\mathrm{online}}_{i}\left(\hat{\mathbf{R}}^{\mathrm{online}}_{W-1}\right)^{\top}. (9)
𝐫s,j=Log⁡((𝐑s+j)⊤​𝐑^s+j←s+W−1​𝐑s+W−1)∈ℝ3.\mathbf{r}_{s,j}=\operatorname{Log}\!\left((\mathbf{R}_{s+j})^{\top}\hat{\mathbf{R}}_{s+j\leftarrow s+W-1}\mathbf{R}_{s+W-1}\right)\in\mathbb{R}^{3}. (10)
Δ​𝝎s+j−Δ​𝝎s+W−1≈𝐫s,j.\Delta\boldsymbol{\omega}_{s+j}-\Delta\boldsymbol{\omega}_{s+W-1}\approx\mathbf{r}_{s,j}. (11)
ws,j=σ2(σ2+‖𝐫s,j‖22)2,σ=5∘,w_{s,j}=\frac{\sigma^{2}}{(\sigma^{2}+\|\mathbf{r}_{s,j}\|_{2}^{2})^{2}},\qquad\sigma=5^{\circ}, (12)
(𝐀⊤​𝐖𝐀)​Δ​𝝎=𝐀⊤​𝐖𝐫.(\mathbf{A}^{\top}\mathbf{W}\mathbf{A})\Delta\boldsymbol{\omega}=\mathbf{A}^{\top}\mathbf{W}\mathbf{r}. (13)
𝐑i←𝐑i​Exp⁡(Δ​𝝎i).\mathbf{R}_{i}\leftarrow\mathbf{R}_{i}\operatorname{Exp}(\Delta\boldsymbol{\omega}_{i}). (14)

Position averaging.

After rotations are fixed, relative translations are rotated into the global frame. Equations 15–17 solve least absolute deviations in camera centers with one center fixed. ADMM uses a sparse Cholesky-accelerated update.

There are no independent per-window scale variables. This preserves learned translation/pointmap scale relationships but also means inconsistent scales cannot be freely absorbed by the optimizer. The recovered poses transform the already predicted local pointmaps.

Equations 15–17
𝐯^s,j=𝐑s+j⊤​𝐭^s+j←s+W−1≈𝐜s+W−1−𝐜s+j.\hat{\mathbf{v}}_{s,j}=\mathbf{R}_{s+j}^{\top}\hat{\mathbf{t}}_{s+j\leftarrow s+W-1}\approx\mathbf{c}_{s+W-1}-\mathbf{c}_{s+j}. (15)
𝐜s+j−𝐜s+W−1+𝐯^s,j=𝟎,j=0,…,W−2.\mathbf{c}_{s+j}-\mathbf{c}_{s+W-1}+\hat{\mathbf{v}}_{s,j}=\mathbf{0},\qquad j=0,\dots,W-2. (16)
min{𝐜i}∑s=1S∑j=0W−2‖𝐜s+j−𝐜s+W−1+𝐯^s,j‖1,s.t.𝐜W−1=𝟎.\min_{\{\mathbf{c}_{i}\}}\sum_{s=1}^{S}\sum_{j=0}^{W-2}\left\|\mathbf{c}_{s+j}-\mathbf{c}_{s+W-1}+\hat{\mathbf{v}}_{s,j}\right\|_{1},\qquad\mathrm{s.t.}\quad\mathbf{c}_{W-1}=\mathbf{0}. (17)
Complete original PDF block alg2
Algorithm 2 Offline Motion Averaging
Original selectable algorithm
1: Window-relative poses {𝐑^s+j←s+W−1,𝐭^s+j←s+W−1}\{\hat{\mathbf{R}}_{s+j\leftarrow s+W-1},\hat{\mathbf{t}}_{s+j\leftarrow s+W-1}\}, online rotations {𝐑^ionline}\{\hat{\mathbf{R}}^{\mathrm{online}}_{i}\}, window size WW
2: Refined global rotations {𝐑i}\{\mathbf{R}_{i}\} and camera centers {𝐜i}\{\mathbf{c}_{i}\}
3: Initialize 𝐑i←𝐑^ionline​(𝐑^W−1online)⊤\mathbf{R}_{i}\leftarrow\hat{\mathbf{R}}^{\mathrm{online}}_{i}(\hat{\mathbf{R}}^{\mathrm{online}}_{W-1})^{\top}
4: Build sparse rotation incidence matrix 𝐀\mathbf{A} with gauge fixed at frame W−1W-1
5: for k=1k=1 to KrotK_{\mathrm{rot}} do
6:   Compute Lie residuals 𝐫s,j←Log⁡((𝐑s+j)⊤​𝐑^s+j←s+W−1​𝐑s+W−1)\mathbf{r}_{s,j}\leftarrow\operatorname{Log}((\mathbf{R}_{s+j})^{\top}\hat{\mathbf{R}}_{s+j\leftarrow s+W-1}\mathbf{R}_{s+W-1})
7:   Compute robust weights ws,j←σ2/(σ2+‖𝐫s,j‖22)2w_{s,j}\leftarrow\sigma^{2}/(\sigma^{2}+\|\mathbf{r}_{s,j}\|_{2}^{2})^{2}
8:   Solve (𝐀⊤​𝐖𝐀)​Δ​𝝎=𝐀⊤​𝐖𝐫(\mathbf{A}^{\top}\mathbf{W}\mathbf{A})\Delta\boldsymbol{\omega}=\mathbf{A}^{\top}\mathbf{W}\mathbf{r}
9:   Update 𝐑i←𝐑i​Exp⁡(Δ​𝝎i)\mathbf{R}_{i}\leftarrow\mathbf{R}_{i}\operatorname{Exp}(\Delta\boldsymbol{\omega}_{i}) for all frames
10:   if 1F​∑i‖Δ​𝝎i‖2<ϵrot\frac{1}{F}\sum_{i}\|\Delta\boldsymbol{\omega}_{i}\|_{2}<\epsilon_{\mathrm{rot}} then
11:    break
12:   end if
13: end for
14: Rotate relative translations into the global frame: 𝐯^s,j←𝐑s+j⊤​𝐭^s+j←s+W−1\hat{\mathbf{v}}_{s,j}\leftarrow\mathbf{R}_{s+j}^{\top}\hat{\mathbf{t}}_{s+j\leftarrow s+W-1}
15: Build sparse LAD system over camera centers {𝐜i}\{\mathbf{c}_{i}\} with fixed scale, enforcing 𝐜s+j−𝐜s+W−1+𝐯^s,j≈𝟎\mathbf{c}_{s+j}-\mathbf{c}_{s+W-1}+\hat{\mathbf{v}}_{s,j}\approx\mathbf{0}
16: Solve min𝐜⁡‖𝐀t​𝐜−𝐛t‖1\min_{\mathbf{c}}\|\mathbf{A}_{t}\mathbf{c}-\mathbf{b}_{t}\|_{1} with ADMM and sparse Cholesky
17: return {𝐑i},{𝐜i}\{\mathbf{R}_{i}\},\{\mathbf{c}_{i}\}

Appendix C Additional Ablation on Online and Offline Motion Averaging

Without any loop edges, offline averaging improves mean KITTI relative rotation error 0.177°→0.166° and translation error 0.147 m→0.136 m. Sequence 02 translation remains 0.234 m at the reported precision. Local redundancy helps, while these metrics do not establish elimination of global drift.

Complete original PDF block A3.T6
Table 6: Additional ablation on KITTI. We compare Anchor3R-Online and Anchor3R-Offline without loop closure on all 11 KITTI Odometry sequences. Offline w/o LC optimizes only the dense window-relative measurements accumulated during streaming inference, without loop-closure reinsertion. Best RRE and RTE values between online and offline variants are highlighted in bold.
Original selectable table
Sequence #Frames Pose-Online Pose-Offline w/o LC
RRE (∘)↓(^{\circ})\downarrow RTE (m)↓(m)\downarrow RRE (∘)↓(^{\circ})\downarrow RTE (m)↓(m)\downarrow
00 4541 0.200 0.084 0.191 0.072
01 1101 0.220 0.334 0.203 0.305
02 4661 0.198 0.234 0.188 0.234
03 801 0.152 0.072 0.137 0.063
04 271 0.129 0.137 0.120 0.116
05 2761 0.156 0.105 0.149 0.096
06 1101 0.141 0.105 0.134 0.087
07 1101 0.174 0.082 0.166 0.075
08 4071 0.182 0.148 0.168 0.144
09 1591 0.212 0.231 0.202 0.229
10 1201 0.180 0.081 0.169 0.070
Avg. – 0.177 0.147 0.166 0.136

Appendix D Controlled Offline Refinement on VBR

Matched refinement improves LongStream on six of seven routes and VGGT on all seven. Anchor3R’s 6/7 wins over VGGT-Off-5 coexist with a worse mean 19.60 versus 16.96. Its difficult ciampino route dominates this difference.

Keyframe spacing 5/10/15 gives mean 19.60/21.80/23.89 m. Most routes stay broadly similar, but ciampino degrades 78.64→92.51→100.87 m. Sparse retrieval is therefore not uniformly harmless.

Complete original PDF block A4.T7
Table 7: Controlled offline refinement on VBR. We apply the same loop-keyframe retrieval and motion-averaging protocol to LongStream, VGGT, and Anchor3R. Off-kk denotes a loop-keyframe interval of kk. Lower ATE is better.
Original selectable table
Method campus_train0 campus_train1 ciampino_train1 colosseo_train0 diag_train0 pincio_train0 spagna_train0 Avg.
LongStream 100.57 105.55 131.78 72.52 32.35 43.47 59.31 77.93
LongStream-Off-5 87.61 99.86 98.64 45.99 8.46 65.84 39.77 63.74
VGGT-Long 118.59 98.21 172.13 39.56 30.80 53.44 50.27 80.43
VGGT-Off-5 12.20 9.85 28.30 20.21 12.84 19.77 15.56 16.96
Anchor3R-Off-5 5.13 3.52 78.64 17.25 5.35 15.56 11.76 19.60
Anchor3R-Off-10 5.02 2.91 92.51 19.15 5.21 16.51 11.27 21.80
Anchor3R-Off-15 5.15 3.62 100.87 21.65 5.04 18.77 12.11 23.89

Appendix E Failure Case Analysis

Figure 6 shows ciampino trajectories without loop constraints. Frequent turns produce orientation errors that redirect subsequent translations; local averaging alone leaves substantial drift. The figure does not show the fully loop-refined result, whose remaining error is separately reported in Table 7.

Complete original figure A5.F6
Figure 6: Failure case on ciampino_train1. We show the online trajectory and offline refinement without loop-closure constraints. Errors are most evident around frequent turns, where orientation errors affect subsequent translation directions and accumulate over the long trajectory.

Appendix F Evaluation Dataset Details

Evaluation uses complete sequences without frame subsampling: all seven VBR routes, 8,815–18,846 frames and up to 5.2 km; all eleven KITTI camera 02 streams; all twelve Oxford front-camera subsets; sequence 01 of each of the seven 7Scenes scenes; and full TUM sequences.

Waymo uses nine held-out segments, 196–199 frames and 42–351 m each: 163453191, 183829460, 315615587, 346181117, 371159869, 405841035, 460417311, 520018670 and 610454533. The dataset participates in training, but these segments do not.

Concept demonstration

Open image ↗