Research map / 3D vision / Gaussian splatting
Understanding while Exploring: Semantics-driven Active Mapping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ActiveSGM, an active semantic mapping framework on a 3D Gaussian Splatting backbone that uses sparse top-k semantic distributions and geometric plus semantic entropy criteria to choose next-best views.
Problem
Active semantic mapping in unknown environments: robots must pick informative viewpoints while building accurate geometric and semantic maps, under noisy semantic predictions and memory limits of high-dimensional semantic representations.
Approach
Combines a 3DGS SLAM backbone (SplaTAM) with a sparse top-k semantic probability representation per Gaussian, a Hellinger/cosine semantic loss on entropy-masked OneFormer pseudo-labels, and an exploration score that multiplies geometric silhouette-based coverage with semantic rendering entropy and a motion penalty.
Key results
- On Replica novel views, ActiveSGM reaches 84.89 average mIoU using 777 steps versus 80.14 for the passive variant over 2000 steps.
- On MP3D, ActiveSGM reaches 65.58 average IoU with predicted labels, above the best GT-supervised baseline Zhang et al. at 42.92.
- On MP3D 3D reconstruction, ActiveSGM achieves 1.56 cm accuracy and 97.35% completeness ratio, surpassing ActiveGAMER (1.66 cm, 95.32%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add active exploration to semantic mapping | ActiveSGM on Replica novel views · Replica novel views | Avg. mIoU (%): 80.14 -> 84.89 | Helped |
| Add active exploration to semantic mapping | ActiveSGM on Replica novel views · Replica novel views | F-1 (%): 67.81 -> 77.56 | Helped |
| Add active exploration to semantic mapping | ActiveSGM on Replica novel views · Replica novel views | Steps: 2000 -> 777 | Helped |
| Retain more categories in the sparse semantic representation (ablation) | ActiveSGM · Replica, office0 and room0 | Avg. mIoU (%): 83.06 -> 83.34 -> 84.08 | Helped |
| Add cosine similarity loss (ablation) | ActiveSGM semantic loss · Replica, office0 and room0 | Avg. mIoU (%): 82.26 -> 84.08 | Helped |
| Add Hellinger distance loss (ablation) | ActiveSGM semantic loss · Replica, office0 and room0 | Avg. mIoU (%): 82.70 -> 84.08 | Helped |
| Replace Hellinger distance with KL divergence (ablation) | ActiveSGM semantic loss · Replica, office0 and room0 | Avg. mIoU (%): 84.08 -> 82.22 | Hurt |
| Use sparse top-k semantic rendering instead of dense class distributions | ActiveSGM on Replica room0 · Replica, room0 semantic rendering | Rendering time: 61 ms -> 3.1 ms | Helped |
| Use active semantic mapping rather than geometry-only active mapping | ActiveSGM on MP3D · MP3D | Accuracy (cm): 1.66 -> 1.56 | Helped |
| Use active semantic mapping rather than geometry-only active mapping | ActiveSGM on MP3D · MP3D | Completeness ratio (%): 95.32 -> 97.35 | Helped |
| Use ActiveSGM rather than ActiveGAMER | 3D reconstruction on Replica · Replica | Accuracy (cm): 1.16 -> 1.19 | Hurt |
| Use ActiveSGM rather than ActiveGAMER | Novel view rendering on Replica · Replica novel view rendering | SSIM: 0.97 -> 0.96 | Hurt |
Surprising findings
- Ablation shows the Hellinger distance outperforms KL divergence, which the authors report suffers gradient vanishing and needed gradient norm clipping.
- Their sparse top-k semantic rendering on room0 reduces rendering time from 61 ms to 3.1 ms versus a dense 102-class representation.
Limitations
- Assumes perfect localization and perfect robot execution of planned trajectories
- Relies on an external OneFormer segmentation model, so segmentation errors propagate; evaluation only in simulation (Habitat)
- Gradients from the semantic loss are blocked from geometry, leaving geometric and semantic optimization decoupled
Experimental setup
Datasets: Replica ReplicaSLAM MP3D
Benchmarks: Replica ReplicaSLAM MP3D
Metrics: average mIoU Top-1 accuracy Top-3 accuracy mAP F-1 accuracy (cm) completeness (cm) completeness ratio PSNR SSIM
Baselines: SGS-SLAM NIDS-SLAM DNS-SLAM SNI-SLAM ActiveGAMER NARUTO SSMI TARE
Builds on: SplaTAM ActiveGAMER SGS-SLAM OneFormer NARUTO
Closest papers by meaning
- Learning to Map for Active Semantic Goal Navigation (ICLR 2022 · similarity 0.87)
- Active3D: Active High-Fidelity 3D Reconstruction via Multi-Level Uncertainty Quantification (AAAI 2026 · similarity 0.87)
- ActiveGAMER: Active GAussian Mapping through Efficient Rendering (CVPR 2025 · similarity 0.85)
- BeliefMapNav: 3D Voxel-Based Belief Map for Zero-Shot Object Navigation (NeurIPS 2025 · similarity 0.85)
- GAMap: Zero-Shot Object Goal Navigation with Multi-Scale Geometric-Affordance Guidance (NeurIPS 2024 · similarity 0.85)
- GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.