At a glance
MetricNet predicts a scalar that converts a navigation policy’s normalized waypoint sequence into metric coordinates. A position controller can then follow the intermediate waypoints rather than drive directly toward one selected endpoint. MetricNav adds depth-based collision and goal-direction guidance to diffusion sampling.
Simulation and five-run physical tests favor metric trajectory execution for route progress. Collision counts do not improve for every base policy, and the work does not provide a formal collision-avoidance guarantee. Project code, models and deployment videos accompany the paper.
I Introduction
Cross-embodiment navigation policies normalize trajectory lengths during training. At deployment, a constant such as maximum speed divided by control frequency sets movement size but does not recover the physical distance between waypoints. Executing only one endpoint can cut across an otherwise curved, obstacle-avoiding path.
MetricNet estimates the missing scale from a sampled trajectory and current image. MetricNav uses the scaled path for geometric costs. The contribution combines this learned conversion with segment execution and a challenging visual-goal benchmark; its benefits should be separated from claims of universal calibration-free or safe navigation.
II Related Work
Related work covers image-based navigation, additional depth/LiDAR cues and generative policies with collision guidance.
II-A Learning-based Navigation
RECON and GNM learn reusable visual navigation, while depth- or LiDAR-assisted methods supply geometric information. Language-conditioned systems address a different interface; this paper focuses on a front-facing camera and an optional goal image.
II-B Generative Navigation Models
NoMaD generates waypoint sequences through diffusion; FlowNav uses conditional flow matching. NaviDiffusor guides diffusion with obstacle costs, while NavDP and other systems introduce critics or traversability cues.
Demonstrations often omit collisions, so plausible imitation samples need not be safe. MetricNet targets an additional issue shared by normalized policies: their unscaled trajectories cannot directly be checked against real-world geometry.
III Overview
The overview distinguishes normalized action generation from recovering physical scale and executing robot controls.
III-A Problem Definition
A policy maps an image history and optional goal image to ego-frame 2D future waypoints. Without a goal it explores. Only a prefix ending at TW is executed before replanning, with TW no larger than the prediction horizon TA.
The unknown scalar is the average physical distance between consecutive waypoints used during training normalization. MetricNet predicts this quantity; it does not directly re-estimate the full path shape or prove its geometric correctness.
III-B Action Generation for Navigation Policies
Image encoders and a transformer build a context vector that conditions a generative action model. Diffusion starts with Gaussian noise and repeatedly predicts/removes noise; training minimizes noise-prediction MSE. Flow matching supplies an alternative base policy. Equations 1–2 retain the original denoising and training definitions.
Equations 1–2
| (1) |
| (2) |
III-C Action Normalization
First divide a segment by its mean waypoint spacing, then convert positions to increments and min–max normalize each coordinate to [−1,1] using global training extrema. Inference undoes coordinate normalization and integrates increments, but the physical segment scale is unavailable and conventionally replaced by a tuned constant.
Equations 3–6 are preserved verbatim. The printed Equation 5 is not the algebraic inverse of Equation 4, and Equation 3’s summation limits versus denominator leave a waypoint-count ambiguity. These are source reproduction issues, not corrected formulas in this edition.
Equations 3–6
| (3) |
| (4) |
| (5) |
| (6) |
III-D Velocity Control
The baseline converts the chosen waypoint’s distance and bearing into linear/angular velocities over one control interval, then clips them to robot limits. At 15 Hz, a normalized unit displacement would request 15 m/s before clipping; a 0.4 m/s maximum motivates the constant 0.4/15 m.
That constant is a controller step size rather than a learned scene scale. Targeting one waypoint also discards intermediate path geometry, creating corner-cutting even if a correctly scaled full trajectory avoids the obstacle.
Equation 7
| (7) |
IV Methodology
The method learns scale, follows a metric waypoint prefix, then optionally uses metric geometry to guide diffusion samples.
IV-A MetricNet
A 1D convolution encodes the unscaled path into 384 dimensions. EfficientNet-B0 image features are projected from 1,280 to 384 dimensions; a frozen Depth-Anything-V2 DINOv2 ViT-S encoder supplies additional depth-related tokens through a trainable residual block.
A three-layer, six-head transformer combines image, depth and trajectory tokens with a classification token. An MLP maps that output to the scalar scale prediction, trained by MSE against mean waypoint spacing. The source specifies 7×7 image and 16×16 depth patches.
Equation 9 prints λ=10⁻³ while the prose describes scaling the loss to millimeters; multiplying an MSE by that factor is not a direct meter-to-millimeter conversion. The original objective is retained and the implementation detail remains ambiguous. No isolated scale-error table or encoder-component ablation is reported.
Equations 8–9
| (8) |
| (9) |
IV-B Position Control
Multiply every normalized waypoint by the predicted scale, then execute the entire prefix from the current pose through TW. Replan before the full horizon to limit exposure to occluded turns or distant obstacles. This preserves intermediate path geometry, but execution between replans is not continuously reactive to new obstacles.
IV-C MetricNav
MetricNav combines the metric path with an estimated local obstacle map and an inferred goal direction. It first samples trajectories to choose a direction, then generates a second guided batch and ranks its candidates.
IV-C 1 Collision Cost
A metric Depth-Anything-V2 ViT-B model plus approximate camera intrinsics converts the current image to a point cloud. RANSAC labels ground points as free and the remaining points as obstacles. A local TSDF supplies nonzero costs in free as well as occupied space so gradients can encourage clearance before contact.
Equation 10 sums costs at each waypoint and two lateral offsets representing the robot’s boundaries. Monocular depth error, calibration and the ground-plane assumption still affect these checks; discrete path costs alone do not establish collision-free continuous robot motion.
Equation 10
| (10) |
IV-C 2 Goal Cost
Cluster the directions of the Nth waypoints from K initial samples using spherical k-means. Select the trajectory nearest the largest cluster’s center as the goal direction. Guided resampling penalizes one minus cosine similarity to that direction.
This assumes the policy’s most common direction is useful for reaching the image goal. It does not localize the goal metrically or ensure that a multimodal sample majority is correct.
Equation 11
| (11) |
IV-C 3 Guidance
Combine goal and collision costs as αFgoal+βFcoll, then subtract their action gradients during late diffusion steps. The reported settings are α=0.5, β=0.01 and step size μ=0.1, tuned for goal following and clearance; guidance is applied only when k<2.
The method improves samples according to approximate local costs. Learned scale and depth errors, finite sampling and goal-direction mistakes remain possible.
Equations 12–13
| (12) |
| (13) |
IV-C 4 Final Action Selection
Rank guided candidates by γ times goal cosine similarity plus (1−γ) times negative collision cost, then execute the highest-scoring one. γ=1 emphasizes the goal, γ=0 clearance, and intermediate values trade them off. This final ranking is separate from the α/β gradient guidance during sampling.
Equation 14
| (14) |
V Experiments
Experiments compare scale choices, control paradigms and guided policies in simulation and on a TurtleBot4.
V-A Training
NoMaD, FlowNav and NaviDiffusor use official implementations and matched data splits. Policies train on synthetic data for simulation and real data for physical deployment, so this is not direct deployment of the synthetic-trained policy without adaptation.
The common prediction horizon is eight waypoints, with TW=3 for the main controller comparison. MetricNet trains on about 1.5 million real/synthetic samples for fifteen epochs, batch size 1,024, AdamW initial learning rate 10⁻⁴ and cosine scheduling on an NVIDIA H200.
V-B Datasets
Real data combine GoStanford, RECON, TartanDrive, SACSoN and SCAND, sampling image histories of length four. Synthetic training uses Habitat; the authors report 16,000 trajectories across 800 Matterport3D indoor scenes, varying fisheye angle over 0°–180° and camera height over 0.25–1 m. Stair trajectories are excluded.
Random waypoint subsampling broadens the scale targets: skips of 1–5 indoors, up to two outdoors and 1–10 synthetically. MetricNet synthetic observations are spaced every 10 cm; policy training uses 25 cm because denser sampling biased predictions toward straight lines.
V-C Simulation Benchmarking
The benchmark uses image-sequence topological maps in twenty environments, routes at least 7.5 m long, matched starts/seeds and TurtleBot4 collision dimensions. The text mentions twenty maps, five seeds and 2,000 repetitions per baseline; maps-per-environment and repetition accounting are not stated unambiguously.
V-D Real-world Deployment
A TurtleBot4 runs ROS2 Humble with a 15 Hz fisheye camera and RTX A500 laptop inference. Metric waypoints use sequential rotate–translate commands; reactive local planners such as the Dynamic Window Approach are deliberately excluded.
Each model receives five runs through an approximately 20 m corridor with turns and obstacles. A frontal collision or inability to recover within ten seconds terminates evaluation. The printed rotation/distance expressions have coordinate/index ambiguities relative to the earlier controller equation, so they should not be treated as a complete executable controller specification.
V-E Experimental Results
Navigation is measured as the maximum fraction of the topological route reached before collision, while exploration measures distance traveled before collision. Thus values labeled “success rate” or “goal reached” primarily report route progress, not necessarily the fraction of wholly successful trials. The physical study additionally counts collisions per run.
V-E 1 Comparing different controller paradigms
Figure 5 shows higher route-progress and exploration distributions when learned scale is paired with position control, across the three base policies. Fixed maximum-speed/frequency scaling performs worse. Difficult shortest-path routes with tight turns reduce the gaps between base policies.
The box plots include both failures and complete routes for every baseline. The improvement concerns distributional performance, not universal success or an isolated proof that scale estimation alone explains all gains.
V-E 2 Real world deployment
Physical route progress rises from 0.76→0.87 for NoMaD, 0.85→0.93 for FlowNav and 0.89→0.93 for NaviDiffusor when moving from fixed-scale velocity control to learned-scale position control. This comparison changes scale and control together.
Collisions per run increase 1.0→1.8 for NoMaD and 1.0→1.2 for FlowNav, while staying 0.8 for NaviDiffusor. MetricNav reaches 0.96 progress with 0.6 collisions per run. Its slower laptop inference prevents a reported physical velocity-control comparison; five runs in one corridor give limited generalization evidence.
V-E 3 Using goal-guidance and metric TSDF
Table II uses metric-scaled waypoints for every method and TW=1 for this guidance comparison, differing from the main TW=3 setting. MetricNav’s velocity-control progress is 0.51 versus 0.46/0.45/0.46 for NoMaD/FlowNav/NaviDiffusor. Under position control it scores 0.61, tying FlowNav and NaviDiffusor and exceeding NoMaD’s 0.59.
V-E 4 Qualitative Results
Figures 6–7 separately illustrate α/β gradient strengths and γ candidate selection. Larger α pulls samples toward the chosen goal, larger β away from obstacles, and γ balances goal similarity against clearance in the final choice. These are selected qualitative examples rather than a quantitative hyperparameter-sensitivity study.
VI Conclusions
Learned waypoint scale makes normalized policy outputs usable by geometric controllers, and executing path prefixes improves route progress in the reported benchmarks. MetricNav adds local geometric guidance with a modest velocity-control gain and tied position-control performance in Table II.
Limitations include monocular geometric uncertainty, a ground-plane assumption, dependence on sample-majority goal direction, limited real testing, unquantified scale-estimation accuracy and inference cost, and lack of reactive handling during segment execution. The authors propose a reactive controller for dynamic obstacles as future work.
Acknowledgments
The authors acknowledge NHR@FAU computing support under BayernKI v106be, Bavarian funding, the Körber European Science Prize and partial BMFTR support through the Robotics Institute Germany.