Research map / 3D vision / NeRF and radiance fields
NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields
Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, Rares Ambrus
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain NeRF-MAE encoder before fine-tuning for 3D OBB detection (ablation) | 3D Swin encoder + RPN head · ScanNet 3D OBB | AP@50: 14.0 -> 17.0 | Improved |
| pretrain NeRF-MAE encoder for semantic voxel labelling (ablation) | 3D Swin encoder · Front3D segmentation | mIOU: 24.9 -> 34.5 | Improved |
| use NeRF-MAE pretraining versus point-based pretraining baselines on NeRF grid | Front3D voxel labelling · Front3D segmentation | mIOU gain from pretraining: +9.6 vs +3.6 (PointM2AE) | Improved |
| add more pretraining scenes (151 to 1515) (ablation) | NeRF-MAE · Front3D 3D OBB | AP@50: 0.45 -> 0.54 | Improved |
| add augmentation during pretraining (ablation) | NeRF-MAE (Front3D only) · Front3D 3D OBB | AP@50: 0.543 -> 0.591 | Improved |
| add HM3D and Hypersim to pretraining data (ablation) | NeRF-MAE · ScanNet 3D OBB | AP@50: 0.155 -> 0.170 | Improved |
| use higher-quality NeRFs for pretraining (18 vs 28 PSNR) (ablation) | NeRF-MAE · Front3D 3D OBB | AP@25: 36% AP25 improvement | Improved |
| add skip connections in the decoder (ablation) | NeRF-MAE · Front3D and ScanNet 3D OBB | AP@25: 0.83 -> 0.85 (Front3D); 0.54 -> 0.57 (ScanNet) | Improved |
| masking ratio 75% versus 65% or 85% (ablation) | NeRF-MAE · 3D OBB | AP@50: 0.519 / 0.512 -> 0.546 | Improved |
| pretrain on NeRF grid, fine-tune with fewer labelled scenes | NeRF-MAE · Front3D 3D OBB | AP@50: 0.42 with 61 scenes vs 0.41 with 122 scenes (NeRF-RPN) | Improved |
| NeRF-MAE versus ImVoxelNet on Front3D AP@25 and AP@50 | NeRF-MAE · Front3D 3D OBB | AP@50: 0.664 -> 0.630 | Hurt |
| NeRF-MAE versus NeRF-RPN on voxel super-resolution | NeRF-MAE · Voxel super-resolution 256^3 | PSNR: 16.25 -> 17.27 | Improved |
Closest papers by meaning
- To View Transform or Not to View Transform: NeRF-based Pre-training Perspective (ICLR 2026 · similarity 0.92)
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.91)
- CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis (AAAI 2025 · similarity 0.89)
- Neural NeRF Compression (ICML 2024 · similarity 0.89)
- NeRFCodec: Neural Feature Compression Meets Neural Radiance Fields for Memory-Efficient Scene Representation (CVPR 2024 · similarity 0.89)
- Bringing NeRFs to the Latent Space: Inverse Graphics Autoencoder (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
