Research map / 3D vision / NeRF and radiance fields
Entity-NeRF: Detecting and Removing Moving Entities in Urban Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Entity-NeRF, which removes moving objects from urban NeRF training by combining entity-wise residual-rank statistics (on CropFormer entity segmentation) with a cooperative thing/stuff stationary-entity classifier, and introduces the MovieMap urban dataset.
Problem
Learning static NeRFs of urban scenes containing many moving objects of varied categories and scales, where explicit dynamic modeling and purely statistical outlier rejection (RobustNeRF) fail.
Approach
Label distractors per entity rather than per pixel or patch by averaging normalized residual ranks over entity segments from a pretrained entity segmentation network, and protect stationary 'stuff' entities (buildings, sky, roads) from being excluded using a thing/stuff classifier fine-tuned per scene.
Key results
- On MovieMap with Mip-NeRF 360, Entity-NeRF reaches 20.74 foreground PSNR versus 20.15 for RobustNeRF, and 25.50 background PSNR versus 22.52.
- On MovieMap with Nerfacto, Entity-NeRF improves foreground PSNR from 17.63 (RobustNeRF) to 19.82 while keeping background PSNR at 24.00 versus 21.74.
- Histogram of per-entity labels is skewed toward 0 or 1, indicating entities are cleanly separated into moving or static.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace RobustNeRF with Entity-NeRF | Nerfacto · MovieMap Dataset | foreground PSNR: 17.63 -> 19.82 | Helped |
| replace RobustNeRF with Entity-NeRF | Nerfacto · MovieMap Dataset | background PSNR: 21.74 -> 24.00 | Helped |
| replace RobustNeRF with Entity-NeRF | Mip-NeRF 360 · MovieMap Dataset | foreground PSNR: 20.15 -> 20.74 | Helped |
| replace RobustNeRF with Entity-NeRF | Mip-NeRF 360 · MovieMap Dataset | background PSNR: 22.52 -> 25.50 | Helped |
| add stationary entity classification (ablation) | Entity-NeRF with Nerfacto · MovieMap Dataset | foreground PSNR: 19.48 -> 19.82 | Helped |
| add stationary entity classification (ablation) | Entity-NeRF with Nerfacto · MovieMap Dataset | background PSNR: 23.68 -> 24.00 | Helped |
| add stationary entity classification (ablation) | Entity-NeRF with Mip-NeRF 360 · MovieMap Dataset | foreground PSNR: 20.20 -> 20.74 | Helped |
| add stationary entity classification (ablation) | Entity-NeRF with Mip-NeRF 360 · MovieMap Dataset | background PSNR: 25.49 -> 25.50 | No clear effect |
Surprising findings
- Purely MSE-trained NeRF gets the highest background PSNR on Mip-NeRF 360 (27.36) yet very low foreground PSNR (11.40), showing overall PSNR is misleading for distractor removal.
Limitations
- Cannot handle shadows cast by moving objects, which may be incorporated into training.
- Large moving objects that hide background regions from other views cannot be reconstructed; inpainting is suggested as a fix.
- Evaluation uses a small dataset of three urban scenes with manual annotation.
Experimental setup
Datasets: MovieMap Dataset RobustNeRF Dataset ADE20K
Benchmarks: MovieMap Dataset RobustNeRF Dataset
Metrics: foreground PSNR background PSNR PSNR SSIM LPIPS
Baselines: RobustNeRF Nerfacto Mip-NeRF 360 D2NeRF RoDynRF
Builds on: RobustNeRF Mip-NeRF 360 Nerfacto CropFormer SegFormer
Closest papers by meaning
- SANeRF-HQ: Segment Anything for NeRF in High Quality (CVPR 2024 · similarity 0.84)
- TFS-NeRF: Template-Free NeRF for Semantic 3D Reconstruction of Dynamic Scene (NeurIPS 2024 · similarity 0.84)
- NeRF-HuGS: Improved Neural Radiance Fields in Non-static Scenes Using Heuristics-Guided Segmentation (CVPR 2024 · similarity 0.83)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.83)
- Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates (CVPR 2024 · similarity 0.83)
- EntitySAM: Segment Everything in Video (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.