Literature
Display
← All papers

MetricNet: Recovering Metric Scale in Generative Navigation Policies

Abhijeet Nayak · Débora Oliveira Makowski · Samiran Gode · Cordelia Schmid · Wolfram Burgard

· ICRA 2026 ↗

Robotics9 min text · CondensedOriginal paper ↗
Contents

At a glance

MetricNet predicts a scalar that converts a navigation policy’s normalized waypoint sequence into metric coordinates. A position controller can then follow the intermediate waypoints rather than drive directly toward one selected endpoint. MetricNav adds depth-based collision and goal-direction guidance to diffusion sampling.

Simulation and five-run physical tests favor metric trajectory execution for route progress. Collision counts do not improve for every base policy, and the work does not provide a formal collision-avoidance guarantee. Project code, models and deployment videos accompany the paper.

I Introduction

Cross-embodiment navigation policies normalize trajectory lengths during training. At deployment, a constant such as maximum speed divided by control frequency sets movement size but does not recover the physical distance between waypoints. Executing only one endpoint can cut across an otherwise curved, obstacle-avoiding path.

MetricNet estimates the missing scale from a sampled trajectory and current image. MetricNav uses the scaled path for geometric costs. The contribution combines this learned conversion with segment execution and a challenging visual-goal benchmark; its benefits should be separated from claims of universal calibration-free or safe navigation.

Complete original Figure 1.
Fig. 1: Two examples of MetricNet transforming waypoints sampled from NoMaD [13] (red) into a collision-free plan (green) around an obstacle to reach the goal. First, our novel MetricNet module grounds the waypoints in the 3D world by predicting a metric scale. Second, MetricNav leverages these scaled points for goal following and collision avoidance around objects. The trajectories are plotted on a Truncated Signed Distance Function (TSDF).
Complete original Figure 2a.
(a) Using different waypoints as goal for the velocity controller.
Complete original Figure 2b.
(b) Scaling the same trajectory with different scales.
Fig. 2: In (a), we illustrate that when using a velocity controller, selecting only the goal waypoint can still lead to collisions despite the predicted trajectory in metric scale being free of obstacles. Note this waypoint only points towards the direction. In (b), we show that depending on the scale factor ϕ\phi, the resulting path may collide with walls if the scale is too small or too large for the environment. This highlights the need not only for the correct scale but also for a controller that executes the whole trajectory in metric space.

II Related Work

Related work covers image-based navigation, additional depth/LiDAR cues and generative policies with collision guidance.

II-A Learning-based Navigation

RECON and GNM learn reusable visual navigation, while depth- or LiDAR-assisted methods supply geometric information. Language-conditioned systems address a different interface; this paper focuses on a front-facing camera and an optional goal image.

II-B Generative Navigation Models

NoMaD generates waypoint sequences through diffusion; FlowNav uses conditional flow matching. NaviDiffusor guides diffusion with obstacle costs, while NavDP and other systems introduce critics or traversability cues.

Demonstrations often omit collisions, so plausible imitation samples need not be safe. MetricNet targets an additional issue shared by normalized policies: their unscaled trajectories cannot directly be checked against real-world geometry.

III Overview

The overview distinguishes normalized action generation from recovering physical scale and executing robot controls.

III-A Problem Definition

A policy maps an image history and optional goal image to ego-frame 2D future waypoints. Without a goal it explores. Only a prefix ending at TW is executed before replanning, with TW no larger than the prediction horizon TA.

The unknown scalar is the average physical distance between consecutive waypoints used during training normalization. MetricNet predicts this quantity; it does not directly re-estimate the full path shape or prove its geometric correctness.

III-B Action Generation for Navigation Policies

Image encoders and a transformer build a context vector that conditions a generative action model. Diffusion starts with Gaussian noise and repeatedly predicts/removes noise; training minimizes noise-prediction MSE. Flow matching supplies an alternative base policy. Equations 1–2 retain the original denoising and training definitions.

Equations 1–2
Aik−1=η⁡(Aik−γ​εθ​(Aik,𝐜i,k)+𝒩⁡(0,σ2​I)).\begin{split}A_{i}^{k-1}=\;&\eta(A^{k}_{i}-\gamma\varepsilon_{\theta}\big(A_{i}^{k},\mathbf{c}_{i},k)+\mathcal{N}(0,\sigma^{2}I)\big).\end{split} (1)
ℒ=MSE​(εk,εθ​(Aik,𝐜i,k)).\mathcal{L}=\text{MSE}\Big(\varepsilon_{k},\varepsilon_{\theta}\big(A_{i}^{k},\mathbf{c}_{i},k)\Big). (2)

III-C Action Normalization

First divide a segment by its mean waypoint spacing, then convert positions to increments and min–max normalize each coordinate to [−1,1] using global training extrema. Inference undoes coordinate normalization and integrates increments, but the physical segment scale is unavailable and conventionally replaced by a tuned constant.

Equations 3–6 are preserved verbatim. The printed Equation 5 is not the algebraic inverse of Equation 4, and Equation 3’s summation limits versus denominator leave a waypoint-count ambiguity. These are source reproduction issues, not corrected formulas in this edition.

Equations 3–6
ϕgt=∑k=0TA−1∥𝐚i+(k+1)−𝐚i+k∥2TA−1.\phi_{\text{gt}}=\frac{\sum_{k=0}^{T_{A}-1}\lVert\mathbf{a}_{i+(k+1)}-\mathbf{a}_{i+k}\rVert_{2}}{T_{A}-1}. (3)
Δ​𝐚^=2​(Δ​𝐚~−[xmin,ymin])[xmax,ymax]−[xmin,ymin]−1,\Delta\hat{\mathbf{a}}=\frac{2\big(\Delta\tilde{\mathbf{a}}-[x_{\text{min}},y_{\text{min}}]\big)}{[x_{\text{max}},y_{\text{max}}]-[x_{\text{min}},y_{\text{min}}]}-1, (4)
Δ​𝐚~=[xmax,ymax]−[xmin,ymin]2​(Δ​𝐚^−[xmin,ymin])+1.\Delta\tilde{\mathbf{a}}=\frac{[x_{\text{max}},y_{\text{max}}]-[x_{\text{min}},y_{\text{min}}]}{2\big(\Delta\hat{\mathbf{a}}-[x_{\text{min}},y_{\text{min}}]\big)}+1. (5)
Ai=ϕtuned​A~i.A_{i}=\phi_{\text{tuned}}\tilde{A}_{i}. (6)

III-D Velocity Control

The baseline converts the chosen waypoint’s distance and bearing into linear/angular velocities over one control interval, then clips them to robot limits. At 15 Hz, a normalized unit displacement would request 15 m/s before clipping; a 0.4 m/s maximum motivates the constant 0.4/15 m.

That constant is a controller step size rather than a learned scene scale. Targeting one waypoint also discards intermediate path geometry, creating corner-cutting even if a correctly scaled full trajectory avoids the obstacle.

Equation 7
vi=∥𝐚i+TW∥2δ​t​ and ωi=arctan⁡(y~i+TW/x~i+TW)δ​t,\begin{split}v_{i}&=\frac{\lVert\mathbf{a}_{i+T_{W}}\rVert_{2}}{\delta t}\text{ and }\\ \omega_{i}&=\frac{\arctan(\tilde{y}_{i+T_{W}}/\tilde{x}_{i+T_{W}})}{\delta t},\end{split} (7)

IV Methodology

The method learns scale, follows a metric waypoint prefix, then optionally uses metric geometry to guide diffusion samples.

IV-A MetricNet

A 1D convolution encodes the unscaled path into 384 dimensions. EfficientNet-B0 image features are projected from 1,280 to 384 dimensions; a frozen Depth-Anything-V2 DINOv2 ViT-S encoder supplies additional depth-related tokens through a trainable residual block.

A three-layer, six-head transformer combines image, depth and trajectory tokens with a classification token. An MLP maps that output to the scalar scale prediction, trained by MSE against mean waypoint spacing. The source specifies 7×7 image and 16×16 depth patches.

Equation 9 prints λ=10⁻³ while the prose describes scaling the loss to millimeters; multiplying an MSE by that factor is not a direct meter-to-millimeter conversion. The original objective is retained and the implementation detail remains ambiguous. No isolated scale-error table or encoder-component ablation is reported.

Complete original Figure 3.
Fig. 3: MetricNet architecture. Our network estimates a factor that converts the unscaled output of generative visual-goal navigation policies into metric scale. MetricNet first tokenizes patches of the current observation using an image encoder. In parallel, the observation is processed with the pre-trained encoder from the Depth-Anything-V2 [31] to produce depth patch tokens. A transformer then processes the combined token sequence, prepending a CLS token to summarize an aggregated representation. Finally, the output CLS token is passed through an MLP to predict the estimated scale. This scale is multiplied by the original unscaled waypoints (blue), generating grounded trajectories in metric coordinates (red).
Equations 8–9
CLSout=F⁡(CLSin,wθ​(Ai),ρθ​(𝐨i),dθ​(𝐨i)).\text{CLS}_{\text{out}}=F\big(\text{CLS}_{\text{in}},w_{\theta}(A_{i}),\rho_{\theta}(\mathbf{o}_{i}),d_{\theta}(\mathbf{o}_{i})\big). (8)
ℒ⁡(α)=λ⋅MSE​(ϕpred,ϕgt),\mathcal{L}(\alpha)=\lambda\cdot\text{MSE}\Big(\phi_{\text{pred}},\phi_{\text{gt}}\Big), (9)

IV-B Position Control

Multiply every normalized waypoint by the predicted scale, then execute the entire prefix from the current pose through TW. Replan before the full horizon to limit exposure to occluded turns or distant obstacles. This preserves intermediate path geometry, but execution between replans is not continuously reactive to new obstacles.

IV-C MetricNav

MetricNav combines the metric path with an estimated local obstacle map and an inferred goal direction. It first samples trajectories to choose a direction, then generates a second guided batch and ranks its candidates.

Complete original Figure 4.
Fig. 4: MetricNav architecture uses a navigation policy to sample 𝒦\mathcal{K} trajectories (blue). Next, spherical k-means clustering is used to estimate the most common goal trajectory (red). This trajectory is then used to derive a cost composed of a goal following and a collision avoidance term using an estimated TSDF from monocular depth. This cost is then fed into the diffusion policy to guide the sampled trajectory into the obstacle-free space, while moving towards the goal. The white arrows show the goal guiding gradients whereas the brown arrows show the collision avoiding gradients.

IV-C 1 Collision Cost

A metric Depth-Anything-V2 ViT-B model plus approximate camera intrinsics converts the current image to a point cloud. RANSAC labels ground points as free and the remaining points as obstacles. A local TSDF supplies nonzero costs in free as well as occupied space so gradients can encourage clearance before contact.

Equation 10 sums costs at each waypoint and two lateral offsets representing the robot’s boundaries. Monocular depth error, calibration and the ground-plane assumption still affect these checks; discrete path costs alone do not establish collision-free continuous robot motion.

Equation 10
ℱcoll(Aik,𝐨i)=∑m=ii+TA[𝒞(𝐚m)+𝒞⁡(𝐚m+σL)+𝒞(𝐚m+σR)],\begin{split}\mathcal{F}_{\text{coll}}\text{}(A_{i}^{k},\mathbf{o}_{i})=\sum_{m=i}^{i+T_{A}}\big[\mathcal{C}(\mathbf{a}_{m})&+\mathcal{C}(\mathbf{a}_{m}+\sigma_{\text{L}})\\ &+\mathcal{C}(\mathbf{a}_{m}+\sigma_{\text{R}})\big],\end{split} (10)

IV-C 2 Goal Cost

Cluster the directions of the Nth waypoints from K initial samples using spherical k-means. Select the trajectory nearest the largest cluster’s center as the goal direction. Guided resampling penalizes one minus cosine similarity to that direction.

This assumes the policy’s most common direction is useful for reaching the image goal. It does not localize the goal metrically or ensure that a multimodal sample majority is correct.

Equation 11
ℱgoal​(Ai)=1−𝐯goal⋅𝐯𝐚i+N‖𝐯goal‖2​‖𝐯𝐚i+N‖2,\mathcal{F}_{\text{goal}}(A_{i})=1-\frac{\mathbf{v}_{\text{goal}}\cdot\mathbf{v}_{\mathbf{a}_{i+N}}}{\|\mathbf{v}_{\text{goal}}\|_{2}\|\mathbf{v}_{\mathbf{a}_{i+N}}\|_{2}}, (11)

IV-C 3 Guidance

Combine goal and collision costs as αFgoal+βFcoll, then subtract their action gradients during late diffusion steps. The reported settings are α=0.5, β=0.01 and step size μ=0.1, tuned for goal following and clearance; guidance is applied only when k<2.

The method improves samples according to approximate local costs. Learned scale and depth errors, finite sampling and goal-direction mistakes remain possible.

Equations 12–13
ℱtotal​(Aik,𝐨i)=α​ℱgoal​(Aik)+β​ℱcoll​(Aik,𝐨i).\mathcal{F}_{\text{total}}(A_{i}^{k},\mathbf{o}_{i})=\alpha\mathcal{F}_{\text{goal}}(A_{i}^{k})+\beta\mathcal{F}_{\text{coll}}(A_{i}^{k},\mathbf{o}_{i}). (12)
Aik−1=η⁡(Aik−γ​εθ​(Aik,𝐜i,k)+𝒩⁡(0,σ2​I))−μ​∇Aikℱtotal​(Aik,𝐨i).\begin{split}A_{i}^{k-1}=\;&\eta(A^{k}_{i}-\gamma\varepsilon_{\theta}\big(A_{i}^{k},\mathbf{c}_{i},k)+\mathcal{N}(0,\sigma^{2}I)\big)\\ &-\mu\nabla_{A_{i}^{k}}\mathcal{F}_{\text{total}}(A_{i}^{k},\mathbf{o}_{i}).\end{split} (13)

IV-C 4 Final Action Selection

Rank guided candidates by γ times goal cosine similarity plus (1−γ) times negative collision cost, then execute the highest-scoring one. γ=1 emphasizes the goal, γ=0 clearance, and intermediate values trade them off. This final ranking is separate from the α/β gradient guidance during sampling.

Equation 14
𝒞total=γ​𝒞G+(1−γ)​𝒞C​A.\mathcal{C}_{\text{total}}=\gamma\mathcal{C}_{G}+(1-\gamma)\mathcal{C}_{CA}. (14)

V Experiments

Experiments compare scale choices, control paradigms and guided policies in simulation and on a TurtleBot4.

V-A Training

NoMaD, FlowNav and NaviDiffusor use official implementations and matched data splits. Policies train on synthetic data for simulation and real data for physical deployment, so this is not direct deployment of the synthetic-trained policy without adaptation.

The common prediction horizon is eight waypoints, with TW=3 for the main controller comparison. MetricNet trains on about 1.5 million real/synthetic samples for fifteen epochs, batch size 1,024, AdamW initial learning rate 10⁻⁴ and cosine scheduling on an NVIDIA H200.

V-B Datasets

Real data combine GoStanford, RECON, TartanDrive, SACSoN and SCAND, sampling image histories of length four. Synthetic training uses Habitat; the authors report 16,000 trajectories across 800 Matterport3D indoor scenes, varying fisheye angle over 0°–180° and camera height over 0.25–1 m. Stair trajectories are excluded.

Random waypoint subsampling broadens the scale targets: skips of 1–5 indoors, up to two outdoors and 1–10 synthetically. MetricNet synthetic observations are spaced every 10 cm; policy training uses 25 cm because denser sampling biased predictions toward straight lines.

V-C Simulation Benchmarking

The benchmark uses image-sequence topological maps in twenty environments, routes at least 7.5 m long, matched starts/seeds and TurtleBot4 collision dimensions. The text mentions twenty maps, five seeds and 2,000 repetitions per baseline; maps-per-environment and repetition accounting are not stated unambiguously.

V-D Real-world Deployment

A TurtleBot4 runs ROS2 Humble with a 15 Hz fisheye camera and RTX A500 laptop inference. Metric waypoints use sequential rotate–translate commands; reactive local planners such as the Dynamic Window Approach are deliberately excluded.

Each model receives five runs through an approximately 20 m corridor with turns and obstacles. A frontal collision or inability to recover within ten seconds terminates evaluation. The printed rotation/distance expressions have coordinate/index ambiguities relative to the earlier controller equation, so they should not be treated as a complete executable controller specification.

V-E Experimental Results

Navigation is measured as the maximum fraction of the topological route reached before collision, while exploration measures distance traveled before collision. Thus values labeled “success rate” or “goal reached” primarily report route progress, not necessarily the fraction of wholly successful trials. The physical study additionally counts collisions per run.

Complete original Figure 5.
Fig. 5: Box plot for navigation and exploration in simulation across velocity and position control using a constant scale and prediction by MetricNet. Using MetricNet outperforms using the constant scale proposed by previous work across all base navigation policies. Moreover, using metric waypoints predicted by MetricNet with position controller improves the result compared to the velocity control. Note that in all navigation experiments, at least one seed fails and one reaches the goal across all baseline policies. The key difference lies in the median of the distribution and in how many average seeds succeed.

V-E 1 Comparing different controller paradigms

Figure 5 shows higher route-progress and exploration distributions when learned scale is paired with position control, across the three base policies. Fixed maximum-speed/frequency scaling performs worse. Difficult shortest-path routes with tight turns reduce the gaps between base policies.

The box plots include both failures and complete routes for every baseline. The improvement concerns distributional performance, not universal success or an isolated proof that scale estimation alone explains all gains.

TABLE I: Success rate and no. of collisions per run for navigation in real-world deployment
Method Velocity control Position control
SR ↑\uparrow #Coll ↓\downarrow SR ↑\uparrow #Coll ↓\downarrow
NoMaD [13] 0.76 1.0 0.87 1.8
FlowNav [11] 0.85 1.0 0.93 1.2
NaviDiffusor [20] 0.89 0.8 0.93 0.8
MetricNav (ours) - - 0.96 0.6
TABLE II: Average fraction of the topomap completed using position and velocity control in the simulation benchmark
Method Goal reached ↑\uparrow
NoMaD FlowNav NaviDiffusor MetricNav
Vel. control 0.46 0.45 0.46 0.51
Pos. control 0.59 0.61 0.61 0.61

V-E 2 Real world deployment

Physical route progress rises from 0.76→0.87 for NoMaD, 0.85→0.93 for FlowNav and 0.89→0.93 for NaviDiffusor when moving from fixed-scale velocity control to learned-scale position control. This comparison changes scale and control together.

Collisions per run increase 1.0→1.8 for NoMaD and 1.0→1.2 for FlowNav, while staying 0.8 for NaviDiffusor. MetricNav reaches 0.96 progress with 0.6 collisions per run. Its slower laptop inference prevents a reported physical velocity-control comparison; five runs in one corridor give limited generalization evidence.

Refer to caption
Fig. 6: Example of trajectories generated with different values of α\alpha and β\beta. The larger the value of α\alpha, the stronger the guidance of the denoised trajectories to the goal. The larger the value of β\beta, the stronger the guidance away from obstacles.
Refer to caption
Fig. 7: Example of trajectories produced with different γ\gamma. The original selected goal trajectory (orange) collides with the object, whereas all guided and scaled trajectories by MetricNav lie in the free space. The trajectory with γ=1\gamma=1 (blue) considers only goal cost and stays close to the goal. The trajectory with γ=0\gamma=0 (red) considers only collision avoidance and prefers driving itself away to a safe state. A balance can be found with γ=0.5\gamma=0.5 (green).

V-E 3 Using goal-guidance and metric TSDF

Table II uses metric-scaled waypoints for every method and TW=1 for this guidance comparison, differing from the main TW=3 setting. MetricNav’s velocity-control progress is 0.51 versus 0.46/0.45/0.46 for NoMaD/FlowNav/NaviDiffusor. Under position control it scores 0.61, tying FlowNav and NaviDiffusor and exceeding NoMaD’s 0.59.

V-E 4 Qualitative Results

Figures 6–7 separately illustrate α/β gradient strengths and γ candidate selection. Larger α pulls samples toward the chosen goal, larger β away from obstacles, and γ balances goal similarity against clearance in the final choice. These are selected qualitative examples rather than a quantitative hyperparameter-sensitivity study.

VI Conclusions

Learned waypoint scale makes normalized policy outputs usable by geometric controllers, and executing path prefixes improves route progress in the reported benchmarks. MetricNav adds local geometric guidance with a modest velocity-control gain and tied position-control performance in Table II.

Limitations include monocular geometric uncertainty, a ground-plane assumption, dependence on sample-majority goal direction, limited real testing, unquantified scale-estimation accuracy and inference cost, and lack of reactive handling during segment execution. The authors propose a reactive controller for dynamic obstacles as future work.

Acknowledgments

The authors acknowledge NHR@FAU computing support under BayernKI v106be, Bavarian funding, the Körber European Science Prize and partial BMFTR support through the Robotics Institute Germany.

Concept demonstration

Open image ↗