Literature
Display
← All papers

FlowNav: Combining Flow Matching and Depth Priors for Efficient Navigation

Samiran Gode · Abhijeet Nayak · Débora N. P. Oliveira · Michael Krawez · Cordelia Schmid · Wolfram Burgard

· IROS 2025 ↗

Robotics7 min text · CondensedOriginal paper ↗
Contents

At a glance

FlowNav combines conditional flow matching for waypoint generation with visual features from a pretrained monocular depth model. It adapts NoMaD’s shared goal-navigation/exploration architecture and trains on mixed robot-navigation data. TurtleBot4 experiments report fewer collisions and good performance with fewer generative integration steps than the diffusion baseline. The evidence supports these two changes in four indoor environments; the policy still makes collisions and does not provide a collision-avoidance guarantee.

I Introduction

Image-conditioned navigation must infer useful geometry and generate control actions quickly. FlowNav addresses these separately: a depth-model encoder adds geometric priors, while a learned velocity field transports Gaussian samples into waypoint trajectories. The motivation is to reduce the iterative inference burden of diffusion without losing multimodal actions. The model takes monocular RGB observations at deployment, so the depth prior does not require a physical depth sensor. Cross-embodiment refers to the heterogeneous training corpus; the reported physical evaluation uses one robot platform.

Refer to caption
Fig. 1: Comparison of sampled trajectories with goal-masked diffusion (NoMaD) and FlowNav (ours). Sampled trajectories are in blue; yellow trajectories are executed. FlowNav uses a combination of CFM and depth priors to generate collision-free actions.

II Related Work

II-A Classical Navigation

Classical navigation combines mapping/localization with path planning. The paper contrasts their environment-dependent tuning with learned visual policies, but does not experimentally compare FlowNav against a classical planning stack. Its results therefore concern the selected learned baseline rather than demonstrating that maps or geometric planning are generally inferior.

II-B Learning-based Navigation

Prior visual policies learn actions from observation histories and goal images, sometimes normalizing action coordinates to combine multiple robot platforms. Other systems add language, image synthesis, or augmentation. FlowNav follows image-goal navigation and adds features from a depth foundation model; language grounding and reinforcement-learning navigation are outside its scope.

II-C Diffusion-based Navigation Models

NoMaD [10] uses goal masking to share one policy between directed navigation and undirected exploration, while diffusion generates multimodal waypoint sequences. FlowNav retains that organization and replaces the denoising process with flow matching. The intended advantage is satisfactory actions with fewer inference steps, not a lower cost for every network evaluation. Earlier approaches such as ViNT and GNM motivate the design but are not directly re-evaluated in the experiments.

III Overview

III-A Problem Formulation

A policy receives the current frontal camera image and a recent image history, optionally with a goal image, and samples a horizon of future two-dimensional waypoints. The learned distribution supports different possible exploratory routes. Robot-specific control subsequently converts these normalized waypoints into executable motion.

III-B Navigation with Goal Masked Diffusion (NoMaD)

The baseline conditions a noise-prediction U-Net on visual context and repeatedly denoises Gaussian action samples. Its observation and goal encoders feed a four-layer, four-head transformer. FlowNav changes the action objective and adds depth tokens to this shared context, enabling an ablation that separately tests the action generator and perception prior.

Equations 1–2
A(k−1)i=α⁡(Aki−γ​εθ​(Aki,ci,k)+𝒩⁡(0,σ2​I)).\prescript{i}{}{A}_{(k-1)}=\alpha\left(\prescript{i}{}{A}_{k}-\gamma\varepsilon_{\theta}(\prescript{i}{}{A}_{k},\prescript{i}{}{c},k)+\mathcal{N}(0,\sigma^{2}I)\right). (1)
ℒ=MSE​(εk,εθ​(Aki,ci,k)).\mathcal{L}=\text{MSE}\Big(\varepsilon_{k},\varepsilon_{\theta}(\prescript{i}{}{A}_{k},\prescript{i}{}{c},k)\Big). (2)

IV Methodology

The architecture combines observation, optional goal, and depth-model tokens, then uses the resulting context for both temporal-distance prediction and flow-based waypoint sampling.

Refer to caption
Fig. 2: FlowNav Architecture: We follow the architecture used in NoMaD [10]. In contrast to the diffusion policy used for action generation in NOMAD, FlowNav uses Conditional Flow Matching (CFM) which is more robust and efficient than diffusion. Additionally, we use depth priors from off-the-shelf foundation models for monocular depth estimation. FlowNav first tokenizes the current and past observations, along with the goal image, using image encoders. It also process the current observation with the pre-trained DinoV2 encoder from the Depth-Anything-V2 [40] model to generate the depth tokens. A transformer processes the observation, goal and depth tokens to create the observation context embedding ci\prescript{i}{}{c}. We then use the context as input to the temporal distance prediction head fdf_{d} and the velocity prediction network vθv_{\theta}.

IV-A FlowNav Architecture

One EfficientNet-B0 encodes each observation; a second encodes the current image concatenated with the goal image. Goal tokens are masked for exploration. The current image also passes through the ViT-small DINOv2 encoder of Depth Anything V2 [40], producing 384-dimensional features. Adaptive pooling to 64 positions, flattening, and an MLP convert these into depth tokens. A four-layer, four-head transformer integrates all tokens. One MLP predicts temporal distance to the goal, while a one-dimensional conditional U-Net predicts action-space velocity. The system uses depth-pretrained features rather than explicitly reconstructing a metric depth map or checking collision geometry.

IV-B Flow Matching Objective

Flow matching learns a time-varying vector field that transports a simple noise distribution to the target action distribution. The marginal vector field is difficult to supervise directly. Conditional flow matching instead samples endpoint pairs and regresses a tractable conditional velocity; the cited theory [16, 17] gives the same parameter gradient as the marginal objective under its assumptions. FlowNav uses independent noise/data coupling and zero path variance: an intermediate sample is the linear interpolation of its endpoints, with target velocity equal to the data endpoint minus the noise endpoint. Straight training interpolants motivate few-step inference, although the learned marginal flow need not be an exact straight trajectory.

Equations 3–9
d​x=ut​(x)​d​t,dx=u_{t}(x)\,dt, (3)
ℒFM​(θ)=𝔼t∼U⁡(0,1),x∼pt​(x)​‖vθ​(x,t)−ut​(x)‖2.\mathcal{L}_{\text{FM}}(\theta)=\mathbb{E}_{t\sim U(0,1),x\sim p_{t}(x)}\left\|v_{\theta}(x,t)-u_{t}(x)\right\|^{2}. (4)
pt​(x)\displaystyle p_{t}(x) =∫pt​(x∣l)​q​(l)​𝑑l,\displaystyle=\int p_{t}(x\mid l)q(l)\,dl, (5)
ℒCFM​(θ):=𝔼t,q⁡(l),pt​(x∣l)​[‖vθ​(t,x)−ut​(x∣l)‖2],\mathcal{L}_{\text{CFM}}(\theta):=\mathbb{E}_{t,q(l),p_{t}(x\mid l)}\left[\|v_{\theta}(t,x)-u_{t}(x\mid l)\|^{2}\right], (6)
pt​(x∣l)=𝒩⁡(x∣t​x1+(1−t)​x0,σ2),p_{t}(x\mid l)=\mathcal{N}\left(x\mid tx_{1}+\left(1-t\right)x_{0},\sigma^{2}\right), (7)
ut​(x∣l)=x1−x0.u_{t}(x\mid l)=x_{1}-x_{0}. (8)
xt=t​x1+(1−t)​x0.x_{t}=tx_{1}+(1-t)x_{0}. (9)

IV-C FlowNav Policy

At a random training time, the U-Net predicts the velocity from a Gaussian action sequence toward the demonstrated waypoint sequence, conditioned on visual context. Its squared-error loss is added to a weighted temporal-distance regression loss. At inference, Gaussian samples are integrated from time zero to one with first-order Euler updates; the number of updates is the inference-step budget. Output waypoints are unnormalized for the robot controller. Goal-directed operation requires a previously collected topological image map, with temporal-distance estimates used to relate observations and map nodes. Goal-masked exploration produces multimodal local routes and can collect that map. Thus the image-only policy is not evidence of map-free long-range goal navigation.

Equations 10–15
ut​(x∣l)=A1i−A0i.u_{t}(x\mid l)=\prescript{i}{}{A}_{1}-\prescript{i}{}{A}_{0}. (10)
Ati=t​A1i+(1−t)​A0i,\prescript{i}{}{A}_{t}=t\prescript{i}{}{A}_{1}+(1-t)\prescript{i}{}{A}_{0}, (11)
ℒCFM​(θ)\displaystyle\mathcal{L}_{\text{CFM}}(\theta) =𝔼p⁡(Ati∣Oi,A0i,A1i),q⁡(l),t\displaystyle=\mathbb{E}_{p(\prescript{i}{}{A}_{t}\mid\prescript{i}{}{O},\prescript{i}{}{A}_{0},\prescript{i}{}{A}_{1}),q(l),t} (12)
[‖vθ​(t,Ati,ci)−ut​(Ati∣A0i,A1i)‖2].\displaystyle\quad\left[\|v_{\theta}(t,\prescript{i}{}{A}_{t},\prescript{i}{}{c})-u_{t}(\prescript{i}{}{A}_{t}\mid\prescript{i}{}{A}_{0},\prescript{i}{}{A}_{1})\|^{2}\right].
ℒdist​(θ)=λ⋅MSE​(disttemp​(oi,og),fd​(ci)).\mathcal{L}_{\text{dist}}(\theta)=\lambda\cdot\text{MSE}\Big(\text{dist}_{\text{temp}}(\prescript{i}{}{o},\prescript{g}{}{o}),f_{d}(\prescript{i}{}{c})\Big). (13)
ℒ⁡(θ)=ℒCFM​(θ)+ℒdist​(θ).\mathcal{L}(\theta)=\mathcal{L}_{\text{CFM}}(\theta)+\mathcal{L}_{\text{dist}}(\theta). (14)
At+δj=Atj+δ​vθ​(t,Atj,cj).\prescript{j}{}{A}_{t+\delta}=\prescript{j}{}{A}_{t}+\delta v_{\theta}(t,\prescript{j}{}{A}_{t},\prescript{j}{}{c}). (15)

V Experiments

V-A Implementation Details

The comparison trains plain NoMaD, depth-augmented NoMaD, FlowNav without depth, and full FlowNav on the same splits. The prediction horizon is eight waypoints, and the observation-history parameter is three. Training uses an H100, batch size 512, AdamW, and initial learning rate 10⁻⁴. The baseline cosine schedule made flow training unstable, so alternative schedules were explored; half-cosine warmup and one-cycle training performed similarly. Full FlowNav is trained for 30 epochs, while the depth-free version works best at 17. Different selected schedules/epoch counts should be considered when interpreting the architectural ablation.

V-B Datasets

Training combines Go Stanford, RECON, Tartan Drive, SACSoN, and SCAND, providing indoor/outdoor data from different robots. Random trajectory segments provide observation context and future image goals. Actions are normalized as in earlier cross-robot navigation work. The authors retrain the baseline on this same subset, supporting a controlled comparison but using less data than the original released NoMaD model.

V-C Deployment

All physical tests use a TurtleBot4 with a 15 Hz wide-angle camera, ROS 2 Humble communication, and inference on an RTX A500 laptop GPU. Four indoor environments include gentle/tight turns, boxes, chairs, table legs, glass, and clutter. These conditions test several perception difficulties but do not establish outdoor deployment, new physical embodiments, or performance on a broad independently sampled set of buildings.

Refer to caption
Fig. 3: Examples of environments. We include easy (boxes and sofas) and hard obstacles (tables and chairs). Our test environment also involves navigating through glass walls and cluttered environments.

V-D Experimental Results

Navigation is measured by goal-reaching success and minor collisions per run. A frontal collision or becoming stuck for more than ten seconds counts as a major unrecoverable collision. Exploration follows a closed route and records collisions. The paper does not give the number of runs behind each reported fraction or confidence intervals, limiting assessment of statistical uncertainty.

Refer to caption
Fig. 4: Visualizing rollouts of FlowNav in challenging environments. We sample 8 actions which are shown in blue. A randomly selected action (yellow) is executed on the robot. Note that providing the goal image clusters the trajectories together, while in exploration, the trajectories are multi-modal.

V-D 1 Baseline Comparison

With ten inference steps, NoMaD reaches the goal in 80% of reported trials with 0.7 minor collisions per run; adding depth raises success to 90% and reduces collisions to 0.6. Both FlowNav variants report 100% success, but depth reduces navigation collisions from 0.7 to 0.2. In exploration, collisions per run are 5.0 for NoMaD, 1.5 with depth, 4.5 for depth-free FlowNav, and 1.0 for full FlowNav. The large exploration benefit from depth in both generators supports the perception contribution. Full FlowNav remains imperfect: perfect reported goal success coexists with nonzero collisions.

V-D 2 Qualitative Results

Rollouts show multiple sampled trajectories, with a randomly selected yellow trajectory executed. Goal conditioning concentrates trajectories toward a destination; goal masking permits distinct exploratory directions. These examples illustrate the learned distribution, but random sample execution supplies no explicit safety filter or optimal-route selection.

Refer to caption
Fig. 5: Comparison of generated trajectories with NoMaD-with-depth and FlowNav for 𝒦=2,5\mathcal{K}=2,5 and 10 inference steps. FlowNav produces valid trajectories with fewer steps, showing the effectiveness of flow matching.

V-D 3 Training Data Ablation

The original pretrained NoMaD model averages three exploration collisions per run, improving on the five from the retrained subset baseline but remaining above depth-augmented NoMaD’s 1.5 and FlowNav’s one. This suggests that both training-data scale and depth features influence the result. It is an exploration-only check, not a complete matched-data replication of every benchmark.

TABLE I: Success rate and no. of collisions per run for navigation and exploration modes
Method Navigation Exploration
Success ↑\uparrow Coll. / run ↓\downarrow Coll. / run ↓\downarrow
NOMAD [10] 0.8 0.7 5.0
NoMad-with-depth 0.9 0.6 1.5
FlowNav-without-depth (ours) 1.0 0.7 4.5
FlowNav (ours) 1.0 0.2 1.0

V-D 4 Inference Steps Ablation

The step-count ablation uses simpler routes with soft turns and no inserted obstacles. At two steps, depth-augmented NoMaD has zero navigation success, whereas FlowNav reaches 75%; at five steps, the rates are 25% and 100%; at ten, both reach 100%. The corresponding FlowNav navigation collision counts are 0.25, zero, and zero. In exploration, FlowNav succeeds at all three budgets while the baseline improves from zero to 75% to 100%. The evidence supports fewer flow steps in these simplified settings; it does not show that two-step performance matches ten-step diffusion in the original cluttered environments.

TABLE II: Success rate and collisions per run against no. of inference steps for navigation mode
Method 𝒦\mathcal{K} = 2 𝒦\mathcal{K} = 5 𝒦\mathcal{K} = 10
SR ↑\uparrow # C ↓\downarrow SR ↑\uparrow # C ↓\downarrow SR ↑\uparrow # C ↓\downarrow
NoMad-with-depth 0.0 1.0 0.25 1.0 1.0 0.0
FlowNav (ours) 0.75 0.25 1.0 0.0 1.0 0.0
TABLE III: Success rate against the no. of inference steps in exploration mode
Method 𝒦=2\mathcal{K}=2 𝒦=5\mathcal{K}=5 𝒦=10\mathcal{K}=10
SR ↑\uparrow SR ↑\uparrow SR ↑\uparrow
NoMad-with-depth 0 0.75 1.0
FlowNav (ours) 1.0 1.0 1.0

V-D 5 Inference Time

Reported action inference times for NoMaD are 17.2, 26.7, and 44.8 ms at two, five, and ten steps; FlowNav takes 19, 26.4, and 48.6 ms. Costs at equal step counts are similar, with FlowNav sometimes slower. The speed advantage comes from using fewer steps: two-step FlowNav is about 2.4× faster than the table’s ten-step NoMaD result, while the simplified navigation success rates are 75% and 100%, respectively. The table labels NoMaD generically although the performance step ablation uses its depth variant, and it reports action inference rather than a measured complete sensing-to-control latency.

TABLE IV: Inference times (in ms) for different no. of steps
   𝒦\mathcal{K}       2       5       10   
   NOMAD [10]       17.2       26.7       44.8   
   FlowNav (ours)       19       26.4       48.6   

VI Conclusions

FlowNav’s strongest evidence is the combination of fewer collisions with depth-pretrained features and effective few-step flow sampling. Limits are the single physical robot, four indoor environments, unspecified run counts/uncertainty, residual collisions, different chosen training schedules, and a simplified inference-step test. Goal navigation still relies on a collected topological map. Future proposals include reinforcement learning for route selection and learned implicit representations to replace that map; neither is demonstrated here.

Acknowledgments

The authors acknowledge computing resources from NHR@FAU under BayernKI and support from the Körber European Science Prize.

Concept demonstration

Open image ↗