Overview
This edition follows the complete 13-page v3 source dated August 2, 2025. The displayed author list includes all ten v3 authors, including Zebang Shen, Luan Zhang and Lu Zhang, who were absent from the earlier catalog entry. All eight figures, three tables, five equations and 109 references are retained, with native PDF imagery and vector labels.
At a glance
GS-Occ3D builds binary occupancy labels from driving images by optimizing a Gaussian-surfel scene representation, separating static background, ground and moving vehicles. An octree supplies multiple spatial scales; dedicated road initialization and vehicle tracking address weak texture and motion. Reconstructed points are divided into frame-centered regions, motion-aligned and voxelized with visibility handling. The method improves static-scene geometry against the tested baselines, but downstream results are mixed: it trails LiDAR-derived labels substantially on Waymo’s reference labels while gaining two IoU points in a zero-shot nuScenes comparison.
1 Introduction
Camera-based occupancy predictors commonly depend on LiDAR-derived training labels. This paper instead uses multi-view image reconstruction to curate the 3D supervision itself. Its intended benefit is scaling label generation from ordinary camera-equipped vehicles, although the reported experiments use Waymo data rather than a separate consumer-vehicle crowdsourcing collection.
The main difficulty is reconstructing coherent geometry from limited co-visible views, long trajectories, weakly textured roads and moving objects. The proposed representation combines hierarchical background surfels, explicit ground geometry and separately tracked vehicles. “Vision-only” describes the scene reconstruction inputs: SfM geometry, camera poses, pretrained segmentation/tracking and road-height assumptions still supply priors. It does not mean geometry is learned without assumptions or pretrained models.
The authors attempt reconstruction across Waymo, then exclude unreliable ego-static scenes from downstream training and evaluation. Thus the final occupancy experiments cover a filtered subset, not every scene in the original benchmark.
2 Related Works
NeRF and Gaussian-splatting approaches model large scenes efficiently, but image-rendering quality alone does not ensure accurate occupancy geometry [33]. StreetSurf uses implicit surfaces and monocular cues, while surface-oriented Gaussian methods regularize primitives or convert them into meshes [22][27][96]. GS-Occ3D keeps a point/surfel representation for label generation to avoid a required meshing stage.
Occupancy prediction is a separate downstream task: models map camera observations to occupied/free 3D cells. Occ3D supplies LiDAR-derived benchmark labels, while CVT-Occ uses temporal voxel correspondences [71][94]. This paper changes CVT-Occ’s supervision source rather than proposing a new occupancy-prediction architecture. Its quantitative downstream results concern binary geometry, not semantic-category accuracy.
3 Methods
The pipeline reconstructs scene geometry offline, converts it into per-frame visible occupancy labels, and trains a separate prediction model. Figure 2 retains the initialization, octree/ground/vehicle reconstruction, dynamic aggregation, voxelization and downstream training panels. Reconstruction speed should not be confused with online prediction latency.
3.1 Scalable Vision-only Geometry Reconstruction
Detector-free SfM and image matching provide sparse initial points [24][68]. Mask2Former supplies segmentation for separating scene components [14]. Each octree voxel can generate up to m local Gaussian surfels. The number of levels follows the rounded logarithm of the camera-to-point distance ratio; level-L voxel centers quantize sparse points at spacing ε/2ᴸ. Equations 1–2 preserve the exact notation. Adaptive anchor control changes the representation during optimization, and cumulative levels retain both coarse coverage and fine detail [62].
Ground initialization projects camera poses onto the horizontal plane and sets height using the nearest camera plus a fixed mounting-height offset, following a road-surface reconstruction idea [19]. Surfel orientation inherits the nearest camera rotation to accommodate slope. A smoothness penalty encourages neighboring ground surfels to have consistent heights. This makes camera/road geometry a substantive modeling assumption, rather than evidence that weak texture alone determines road shape.
Moving vehicles start from RGB-based 3D boxes and tracked poses supplied by CC-3DT [20]. Points are sampled within each box, and learnable rotation/translation corrections refine the trajectory as in Street Gaussians [87]. Separate object coordinates support motion-aligned aggregation, but reconstruction quality still depends on tracking, boxes and the assumed object model. The paper does not provide a quantitative dynamic-object geometry breakdown.
The objective combines RGB reconstruction, geometric regularization, object separation, road smoothness and sky opacity. RGB uses L1 and D-SSIM; geometric terms flatten Gaussians into surfels and regularize depth distortion and depth/normal agreement. Object-opacity entropy encourages foreground/background separation, and binary cross-entropy supervises sky opacity. All five original equations remain. Numerical loss weights, several initialization parameters and complete preprocessing costs are not specified in the paper, limiting reproduction from the text alone.
Equations 1–5
| (1) |
| (2) |
| (3) |
| (4) |
| (5) |
3.2 Occupancy Labels Curation
Because optimization reconstructs the scene as a batch, the result is first sliced around each camera pose. A LiDAR-like perception range and point-count convention define a synthetic single-sweep cloud. This is a sampling convention, not use of measured LiDAR geometry as reconstruction input.
Dynamic points are transformed into their tracked box coordinate system, aggregated across observations, then placed back into each frame’s box pose. This avoids the motion smearing produced by merging vehicle points directly in world coordinates. Static frames are slices of one shared scene reconstruction; they do not contribute independent complementary scans. Calling them “complete” means drawn from that shared representation, not proof that all occluded surfaces have been recovered.
Voxelization must distinguish unseen space from empty space. The described ray-casting procedure marks the first occupied voxel on each camera ray as observed and treats farther occupied cells as unobserved. This reduces incorrect free-space labels behind occluders. The source does not fully detail all free-space voting, grid-resolution and ray-sampling conventions. Camera visibility remains a limitation of both the labels and evaluation.
4 Experiments
Experiments assess static reconstruction, label-based downstream occupancy prediction and selected representation ablations. All original scene galleries and comparison tables remain, including results that qualify broad superiority claims.
4.1 Geometry Reconstruction Experimental Setup
Geometry is evaluated on Waymo Static-32, comparing only the static component for compatibility with the baselines [69]. Each scene uses five camera views and about 1,000 images in total; those are not 1,000 independent five-camera timestamps. LiDAR points provide the evaluation reference.
Baselines include NeuS, F²-NeRF, StreetSurf, PGSR, 2DGS and GVKF [78][79][22][9][27][64]. Some are LiDAR-assisted and others vision-only, as marked in Table 1. Explicit methods share the initial point cloud, and the indicated baselines receive the proposed ground initialization. Metrics are Chamfer distance for geometry, PSNR for rendering, storage, GPU memory and optimization time. The table does not state a physical unit for its Chamfer-distance values.
4.2 Geometry Reconstruction Result Analysis
Table 1 reports Chamfer distance 0.56 for GS-Occ3D, versus 0.82 for GVKF, 1.23 for 2DGS, 0.90 for the LiDAR-assisted StreetSurf variant and 0.76 for LiDAR-assisted NeuS. Its PSNR is 26.89, storage 80MB, GPU memory 10GB and reported training time 0.8 hours. It is not best on every resource or rendering measure: GVKF uses 65MB, PGSR uses 4GB, and vision-only StreetSurf has PSNR 27.12.
Camera-count consistency across the comparisons is unclear. Table 1’s PGSR and 2DGS results match the three-camera rows of Table 3, whereas its GVKF and GS-Occ3D values match the five-camera rows. The exact controlled comparison should therefore be read from Table 3 rather than assuming all Table 1 entries use the same number of views.
Figures 3–4 show selected detailed structures and distant regions beyond the LiDAR reference coverage. Such regions can increase Chamfer distance even when visually plausible, but visual plausibility does not independently verify their geometry. These examples support complementary coverage, not a general claim that camera reconstruction is more accurate than the reference sensor. Static-32 also does not establish moving-object accuracy or autonomous-driving safety.
| Method | Geometry | Rendering | Efficiency | |||
| CD | PSNR | MB | GB | Time | ||
| Implicit | NeuS* [78] | 0.76 | 13.24 | 170 | 31 | 5.0h |
| -NeRF [79] | 886.77 | 24.70 | 130 | 24 | 0.8h | |
| StreetSurf [22] | 1.02 | 27.12 | 540 | 22 | 1.5h | |
| StreetSurf† [22] | 0.90 | 26.85 | 245 | 21 | 1.5h | |
| Explicit | PGSR‡ [9] | 2.90 | 22.61 | 78 | 4 | 1.5h |
| 2DGS‡ [27] | 1.23 | 25.60 | 83 | 15 | 1.0h | |
| GVKF‡ [64] | 0.82 | 25.87 | 65 | 24 | 2.0h | |
| Ours | 0.56 | 26.89 | 80 | 10 | 0.8h | |
4.3 3D Occupancy Prediction Experimental Setup
The original Occ3D-Waymo split has 798 training and 202 validation scenes. Removing ego-static scenes leaves 637 and 165, respectively. Both supervision alternatives are compared on this filtered setting. Zero-shot transfer uses the original 150-scene Occ3D-nuScenes validation set, without training CVT-Occ on nuScenes in this experiment.
The same occupancy model is trained using either GS-Occ3D’s binary geometry or Occ3D’s LiDAR-derived labels. Evaluation measures binary IoU, F1, precision and recall only in camera-visible voxels. This restricts the claims to visible geometry; it does not evaluate all unobserved volume or validate the claimed richer semantic categories. Downstream optimization details and repeated-run uncertainty are not fully reported.
4.4 3D Occupancy Prediction Result Analysis
Against Occ3D-Waymo validation labels, camera-label training yields 44.7% IoU versus 57.4% for Occ3D-label training, a 12.7 percentage-point deficit. Precision/recall are 58.2/65.9% versus 62.9/87.0%. The text’s “slightly lower” characterization understates this measured gap.
On zero-shot Occ3D-nuScenes, IoU improves from 31.4% to 33.4%, and F1 from 47.8% to 50.1%. The trade-off is substantial: precision increases from 38.8% to 62.5%, while recall falls from 62.1% to 41.8%. Thus the aggregate evidence supports a small IoU gain with more conservative predictions, not uniformly more complete geometry.
When evaluated against the method’s own Waymo validation labels, its model reaches 46.8% IoU versus 41.1% for the Occ3D-trained model. This shows supervision-dependent agreement, not an independent ground-truth advantage. Its training-set results are 50.6% on its own labels and 48.3% on Occ3D labels; learnability does not establish that all generated labels are correct.
Figures 5–7 retain label examples, Waymo predictions and the semantic/rainy-scene demonstrations. The 66-versus-16 category claim refers to categories obtainable from image segmentation, with no semantic occupancy benchmark reported. The rainy example is qualitative, not a systematic adverse-weather sensor comparison. Broader range and richer color/texture can help, while nighttime, exposure, occlusion and missing rear views still limit reconstruction.
| Train Labels | Eval Labels | IoU | F1 | Prec. | Rec. |
| Ours (Waymo) | Occ3D-Val(Waymo) | 44.7 | 61.8 | 58.2 | 65.9 |
| Occ3D (Waymo) | 57.4 | 73.0 | 62.9 | 87.0 | |
| Ours (Waymo) | Occ3D-Val(nuScenes) | 33.4 | 50.1 | 62.5 | 41.8 |
| Occ3D (Waymo) | 31.4 | 47.8 | 38.8 | 62.1 | |
| Ours (Waymo) | Ours-Val(Waymo) | 46.8 | 63.8 | 54.6 | 76.6 |
| Occ3D (Waymo) | 41.1 | 58.3 | 46.7 | 77.6 | |
| Ours (Waymo) | Ours-Train(Waymo) | 50.6 | 67.2 | 54.3 | 88.1 |
| Occ3D-Train(Waymo) | 48.3 | 65.1 | 60.1 | 71.0 |
4.5 Ablation Studies
With five cameras, GS-Occ3D obtains point-cloud Chamfer distance 0.56 versus 0.66 with three cameras; PSNR changes from 26.96 to 26.89. More views improve its reported geometry but not every metric. Other methods respond differently, so the result does not prove arbitrary crowdsourced camera configurations will work equally well.
For PGSR, 2DGS and GVKF, Table 3’s mesh errors exceed their direct point-cloud errors. This supports avoiding meshing for these implementations under sparse observations; GS-Occ3D itself has no mesh result in the table. Point-cloud sampling and representation differ, so it is not a universal result that meshes are inferior.
Figure 8 preserves two with/without-ground examples, showing fewer holes or abnormal structures with dedicated ground surfels. This particular ablation is qualitative. All methods in Table 3 use ground Gaussians, and the paper does not separately quantify every octree, cumulative-level, dynamic-pose and loss component.
| Views | Method | Geometry | Rendering | |
| CD-Pcd | CD-Mesh | PSNR | ||
| 5Cam | PGSR [9] | 3.63 | 4.41 | 19.18 |
| 2DGS [27] | 1.25 | 2.14 | 23.42 | |
| GVKF [64] | 0.82 | 1.22 | 25.87 | |
| Ours | 0.56 | - | 26.89 | |
| 3Cam | PGSR [9] | 2.90 | 3.03 | 22.61 |
| 2DGS [27] | 1.23 | 1.85 | 25.60 | |
| GVKF [64] | 0.87 | 1.02 | 26.22 | |
| Ours | 0.66 | - | 26.96 | |
5 Conclusion
The paper demonstrates an offline image-to-occupancy supervision pipeline with strong static reconstruction results and a modest zero-shot binary-IoU gain. Its scope is limited by camera coverage, low light/exposure, ego-static reconstruction failures, tracking quality and geometric assumptions. No separate consumer-fleet experiment, systematic weather study, semantic occupancy evaluation or driving/planning outcome is provided. More robust reconstruction under difficult motion and lighting, and stronger semantic validation, remain future work.
Acknowledgments
The authors acknowledge China’s National Key Research and Development Program (2022YFB4701900) and the Tsinghua University–Mercedes-Benz Joint Institute for Sustainable Mobility.