Literature
Display
← All papers

UDON: Uncertainty-weighted Distributed Optimization for Multi-Robot Neural Implicit Mapping under Extreme Communication Constraints

Hongrui Zhao · Xunlan Zhou · Boris Ivanovic · Negar Mehr

· ICRA 2026 ↗

Robotics4 min text · CondensedOriginal paper ↗
Contents

At a glance

UDON makes distributed neural mapping more robust to intermittent communication by separating consensus penalties for each pair of agents and weighting map parameters by observation frequency. Only currently communicating neighbors contribute to an update. Experiments on indoor datasets and robot-collected data show better reconstruction than the tested baselines at low simulated communication success rates, including 1%. Inputs include camera poses; this is primarily a map-fusion study, not a demonstrated solution to unconstrained joint localization.

I INTRODUCTION

Neural maps cannot be merged as simply as point clouds. Consensus optimization must reconcile learned parameters, and stale disagreement penalties can destabilize training after communication drops. UDON replaces each agent’s aggregate dual variable with one variable per communication edge. Combined with uncertainty weighting, this avoids applying penalties from disconnected neighbors while preserving reliable local detail. The restored opening figure compares three robot trajectories and resulting maps.

Three robot trajectories and RAMEN, MACIM, and UDON map reconstructions.
Fig. 1: UDON outperforms existing baselines in a three-robot mapping experiment conducted with a challenging 5% communication success rate. We conducted the experiment using a dataset collected with TurtleBot robots. For visualization purposes, the trajectory of each agent is color-coded. While the baseline methods either fail to converge or produce incomplete reconstructions, UDON, in contrast, successfully fuses information from all agents to create a complete map of the scene.

II RELATED WORKS

Single-agent implicit mapping supplies the compact representation. Prior distributed methods use consensus ADMM, consistency losses, or uncertainty weighting; their tradeoffs include instability under long disconnections and loss of fine detail. UDON combines RAMEN’s update-frequency weighting with an edge-based formulation inspired by MESA. Its contribution is the adaptation and evaluation of this optimization structure for neural mapping.

III Preliminaries

Agents receive posed RGB-D images, maintain private observations, and exchange neural-map parameters with available neighbors. Standard consensus ADMM alternates a dual update that accumulates map disagreement with a local primal optimization balancing reconstruction and consensus. Each local minimization uses a fixed number of gradient steps. A single accumulated dual variable can retain influence from neighbors no longer communicating, motivating the revised formulation.

Equations 1–2
pit+1=pit+ρ​∑j∈𝒩it(Θit−Θjt),p_{i}^{t+1}=p_{i}^{t}+\rho\sum_{j\in\mathcal{N}_{i}^{t}}(\Theta_{i}^{t}-\Theta_{j}^{t}), (1)
Θit+1=\displaystyle\Theta_{i}^{t+1}= argminΘiLio​b​j​(Θi,ℛit)+⟨Θi,pit⟩\displaystyle\operatornamewithlimits{argmin}_{\Theta_{i}}\;L^{obj}_{i}(\Theta_{i},\mathcal{R}_{i}^{t})+\langle\Theta_{i},p_{i}^{t}\rangle
+ρ∑j∈Ni‖Θi−Θit+Θjt2‖22,\displaystyle+\rho\sum_{j\in N_{i}}\left\|\Theta_{i}-\frac{\Theta_{i}^{t}+\Theta_{j}^{t}}{2}\right\|_{2}^{2}, (2)

IV Method

The method combines a shared neural representation, a frequency-based reliability estimate, and pair-specific consensus updates. Algorithm 1 retains the full decentralized update sequence.

IV-A Neural Implicit Representation

Each agent uses a multiresolution hash grid with geometry and color decoders adapted from Co-SLAM. Decoders are pretrained on a NICE-SLAM apartment dataset and then frozen; only hash-grid parameters change during mapping. One-blob coordinate encoding and volumetric ray rendering produce color and depth. Training combines color/depth reconstruction, smoothness, and free-space losses. Shared fixed decoders help make corresponding grid parameters comparable across agents.

IV-B Frequency-based Uncertainty

A parameter’s count of nonzero-gradient updates acts as an observation-frequency proxy: more updates indicate more reliable coverage. Pairwise normalization turns those counts into diagonal consensus weights. Although the paper calls the count vector “uncertainty,” larger counts actually mean lower uncertainty; one prose sentence reverses that interpretation. The mechanism is a heuristic based on sampling frequency, not a calibrated estimate of geometric error or a posterior variance.

Equations 3–4d
uit=∑k=0tsgn​(abs​(∂Lio​b​j​(Θik,ℛik)∂Θik)).u_{i}^{t}=\sum_{k=0}^{t}\text{sgn}\left(\text{abs}\left(\frac{\partial L^{obj}_{i}(\Theta_{i}^{k},\mathcal{R}_{i}^{k})}{\partial\Theta_{i}^{k}}\right)\right). (3)
us​u​m=uit+ujt,\displaystyle u_{sum}=u_{i}^{t}+u_{j}^{t}, (4a)
ϵ=βu−βlmax​(us​u​m)−min​(us​u​m),\displaystyle\epsilon=\frac{\beta_{u}-\beta_{l}}{\text{max}(u_{sum})-\text{min}(u_{sum})}, (4b)
ζ=βl−ϵ⋅min​(us​u​m),\displaystyle\zeta=\beta_{l}-\epsilon\cdot\text{min}(u_{sum}), (4c)
Wi​jt=diag​(ϵ⋅uit+ζ),Wj​it=diag​(ϵ⋅ujt+ζ).\displaystyle W_{ij}^{t}=\text{diag}(\epsilon\cdot u_{i}^{t}+\zeta),\quad W_{ji}^{t}=\text{diag}(\epsilon\cdot u_{j}^{t}+\zeta). (4d)

IV-C Uncertainty-weighted Distributed Optimization

UDON stores a separate dual variable for every agent pair. It updates that variable only when the pair communicates and holds it fixed otherwise. Both dual penalties and weighted quadratic consensus terms in the next primal update use only active neighbors. This prevents a disconnected agent’s old penalty from continually dominating current reconstruction. The paper demonstrates improved empirical stability but does not supply a general convergence guarantee for this nonconvex online setting.

Equations 5–6
p(i,j)t+1=p(i,j)t+2​ρ​Wi​jt​(Wi​jt+Wj​it)−1​(Wj​it​Θit+1−Wj​it​Θjt+1),p_{(i,j)}^{t+1}=p_{(i,j)}^{t}+2\rho W_{ij}^{t}(W_{ij}^{t}+W_{ji}^{t})^{-1}(W_{ji}^{t}\Theta_{i}^{t+1}-W_{ji}^{t}\Theta_{j}^{t+1}), (5)
Θit+1=argminΘiLio​b​j​(Θi,ℛit)+∑j∈𝒩it⟨Θi,p(i,j)t+1⟩\displaystyle\Theta_{i}^{t+1}=\operatornamewithlimits{argmin}_{\Theta_{i}}\;L^{obj}_{i}(\Theta_{i},\mathcal{R}_{i}^{t})+\sum_{j\in\mathcal{N}_{i}^{t}}\langle\Theta_{i},p_{(i,j)}^{t+1}\rangle
+ρ∑j∈𝒩it∥Θi−(Wi​jt+Wj​it)−1(Wi​jtΘit+Wj​itΘjt)∥Wi​jt2,\displaystyle+\rho\sum_{j\in\mathcal{N}_{i}^{t}}\big\|\Theta_{i}-(W_{ij}^{t}+W_{ji}^{t})^{-1}(W_{ij}^{t}\Theta_{i}^{t}+W_{ji}^{t}\Theta_{j}^{t})\big\|_{W_{ij}^{t}}^{2}, (6)
Algorithm 1 UDON
1: Initialization:
2: for i∈𝒱i\in\mathcal{V} do
3:   Θi0=Θinitial\Theta_{i}^{0}=\Theta_{\text{initial}}
4:   for j∈𝒱,j≠ij\in\mathcal{V},j\neq i do
5:    p(i,j)0=0p_{(i,j)}^{0}=0 ⊳\triangleright Initialize all dual variables
6:   end for
7: end for
8:
9: while true do ⊳\triangleright Main Mapping Loop for Agent ii
10:   Send map Θit\Theta_{i}^{t} & uncertainty uitu_{i}^{t} to neighbors 𝒩it\mathcal{N}_{i}^{t}
11:   Receive Θjt,ujt\Theta_{j}^{t},u_{j}^{t} from neighbors j∈𝒩itj\in\mathcal{N}_{i}^{t}
12:   Compute weights Wi​jt,Wj​itW_{ij}^{t},W_{ji}^{t} for all j∈𝒩itj\in\mathcal{N}_{i}^{t} using (4)
13:
14:   for j∈𝒩itj\in\mathcal{N}_{i}^{t} do ⊳\triangleright Dual Update
15:    Update p(i,j)t+1p_{(i,j)}^{t+1} using dual update rule (5)
16:   end for
17:
18:   for b←1​to​Bb\leftarrow 1\;\text{to}\;B do ⊳\triangleright Primal Update
19:    Sample a mini-batch from ℛit\mathcal{R}_{i}^{t}
20:    Update Θi\Theta_{i} via (6) to approximate Θit+1\Theta_{i}^{t+1}
21:   end for
22:
23:   Compute uncertainty uit+1u_{i}^{t+1} using (3)
24: end while

V EXPERIMENTAL RESULTS

Comparisons use a common map representation on Replica, ScanNet, and a TurtleBot dataset, with three independent runs per setting. Communication attempts occur every mapping iteration and are randomly dropped to reach specified success rates; a 1% rate corresponds to about one successful neighbor update per 100 attempts. That probability is not itself a byte-bandwidth or latency measurement. Single-agent runtime is 5.3 frames per second and about 3.8 GB on an RTX 5090. Mesh metrics measure artifacts, missing coverage, and the fraction of ground truth within 5 cm of the reconstruction.

Refer to caption

Fig. 2: Qualitative comparison on ScanNet scene0169 with a challenging 1% communication rate. The agent trajectories are color-coded on the ground truth mesh for reference. Both Di-NeRF and RAMEN completely fail to reconstruct the scene. In contrast, MACIM and UDON both produce coherent maps, demonstrating the significant benefit of even minimal communication over a no-communication baseline (map learned from only blue agent’s observations). However, a closer inspection (highlighted in orange) reveals UDON’s superior ability to capture fine details, accurately reconstructing the chairs against the wall which MACIM misses.
TABLE I: UDON outperforms the state-of-the-art methods at only 1% communication success rate with three agents.
Method Metric office1 room1 scene0000 scene0169
No Communication Artifacts ↓\downarrow 13.76 ±\pm 1.64 4.69 ±\pm 2.36 17.25 ±\pm 11.42 24.44 ±\pm 6.23
Holes ↓\downarrow 20.11 ±\pm 4.94 6.90 ±\pm 2.43 6.42 ±\pm 2.67 6.30 ±\pm 3.07
Completion Ratio (%) ↑\uparrow 51.64 ±\pm 3.82 75.59 ±\pm 8.73 71.80 ±\pm 7.31 76.39 ±\pm 7.17
Di-NeRF [8] Artifacts ↓\downarrow 28.85±\pm11.57 63.29±\pm5.75 52.27±\pm2.65 44.08 ±\pm 9.72
Holes ↓\downarrow 16.96±\pm12.19 67.52±\pm25.92 33.53±\pm12.69 10.35 ±\pm 1.69
Completion Ratio (%) ↑\uparrow 48.87±\pm19.58 4.49±\pm3.24 9.41±\pm6.06 49.42 ±\pm 5.26
MACIM [10] Artifacts ↓\downarrow 7.15±\pm6.27 2.19±\pm2.23 5.53±\pm4.27 21.74 ±\pm 6.99
Holes ↓\downarrow 5.84±\pm6.48 3.37±\pm0.55 3.81±\pm0.79 3.77 ±\pm 1.13
Completion Ratio (%) ↑\uparrow 80.48±\pm11.09 87.49±\pm3.66 83.89±\pm6.42 83.12 ±\pm 4.78
RAMEN [9] Artifacts ↓\downarrow 44.37±\pm11.68 48.12±\pm10.80 56.54±\pm5.38 35.89 ±\pm 3.37
Holes ↓\downarrow 33.55±\pm16.16 51.46±\pm36.18 39.29±\pm9.50 27.59 ±\pm 7.13
Completion Ratio (%) ↑\uparrow 24.93±\pm16.10 15.52±\pm13.08 6.46±\pm2.93 14.46 ±\pm 6.62
UDON (ours) Artifacts ↓\downarrow 4.40±\pm3.27 1.37±\pm0.05 2.83±\pm0.20 20.83 ±\pm 5.28
Holes ↓\downarrow 3.44±\pm0.90 3.11±\pm0.08 3.14±\pm0.26 3.30 ±\pm 1.28
Completion Ratio (%) ↑\uparrow 84.35±\pm6.76 89.46±\pm0.24 90.67±\pm2.89 85.38 ±\pm 6.13

Original figure

Fig. 3: Comparison of reconstruction quality on scene0000 across four communication success rates (1%, 5%, 10%, 20%). UDON (ours) achieves the lowest artifacts and holes and the highest completion ratio, with small variance. MACIM performs reasonably but falls behind UDON as communication degrades, while Di-NeRF and RAMEN collapse. These results highlight UDON’s robustness under constrained communication.

Original figure

Fig. 4: Comparison of reconstruction quality on scene0000 with varying numbers of communicating agents (4, 6, 8) at 5% communication success rate. UDON (ours) consistently outperforms MACIM. Furthermore, reconstruction quality improves as the number of participating agents increases, indicating that even under low communication rates, leveraging more agents provides clear benefits.

V-A Replica and ScanNet Datasets

At 1% communication with three agents, UDON reconstructs two Replica and two ScanNet scenes more accurately than the listed baselines; Di-NeRF and RAMEN become unstable. A sweep from 20% to 1% on scene0000 shows graceful degradation relative to those methods. Increasing agents from four to eight at 5% improves coverage, though UDON has more artifacts than MACIM in the four-agent setting because of floaters in unobserved regions. Thus the figure caption’s blanket superiority claim has this explicit exception. Loss curves support the proposed mechanism: aggregate dual terms diverge while UDON keeps their influence bounded in the tested runs.

Refer to caption

Fig. 5: Qualitative comparison on ScanNet scene0000 with an increasing number of agents (4, 6, 8 agents) at 5% communication success rate. As more agents are added, UDON effectively leverages the additional data to produce increasingly complete scene reconstructions. Compared to MACIM, UDON demonstrates a significant advantage in areas observed by only a few agents (highlighted regions, color-coded by the corresponding agent’s trajectory). Furthermore, UDON captures color and texture with higher fidelity as evidenced by the accurately-reconstructed floor tiles.

Refer to caption

Fig. 6: Analysis of the primal update loss curves for different methods. The total loss, defined in (6), is composed of the Reconstruction Loss, Dual Variable Loss (consensus enforcement), and L2 Consensus Loss (quadratic penalty). The dual variable loss for Di-NeRF and the RAMEN diverges, causing training instability and degrading reconstruction quality. In contrast, UDON stabilizes this loss term. This stability allows UDON to achieve a consistently low reconstruction loss.

V-B Turtlebot Hardware Dataset

Three robot-collected scenes are segmented to simulate three agents at 5% communication. Ground-truth poses come from VICON, and a learned model enhances depth. UDON’s completion ratios are 78.75%, 83.64%, and 80.21%, versus 66.52%, 75.35%, and 65.69% for MACIM, alongside lower artifact and hole distances. This tests map fusion with obstructed robot viewpoints and simulated dropouts; it does not establish end-to-end decentralized tracking under measured radio failures.

TABLE II: Performance Comparison on the TurtleBot Dataset.
Method Metric exp1 exp2 exp3
MACIM [10] Artifacts ↓\downarrow 6.91 ±\pm 0.80 5.92 ±\pm 0.70 5.64 ±\pm 1.25
Holes ↓\downarrow 5.64 ±\pm 0.80 4.36 ±\pm 0.16 5.61 ±\pm 0.38
Completion Ratio (%) ↑\uparrow 66.52 ±\pm 3.54 75.35 ±\pm 0.67 65.69 ±\pm 3.16
UDON (ours) Artifacts ↓\downarrow 5.24 ±\pm 0.68 4.86 ±\pm 0.58 4.58 ±\pm 0.42
Holes ↓\downarrow 3.92 ±\pm 0.58 3.29 ±\pm 0.38 3.60 ±\pm 0.38
Completion Ratio (%) ↑\uparrow 78.75 ±\pm 4.41 83.64 ±\pm 3.15 80.21 ±\pm 3.35

VI CONCLUSIONS

Pair-specific consensus and reliability weighting improve sparse-communication mapping in these indoor experiments. Large-scale maps with incrementally added submaps remain future work. Practical limits include supplied poses, fixed pretrained decoders, heuristic uncertainty, simulated communication probabilities, and evaluation on a small set of scenes.

Acknowledgments

The work acknowledges NSF support and GPU resources from the NVIDIA Academic Grant.

Concept demonstration

Open image ↗