AI Paper Atlas

Research map / 3D vision / NeRF and radiance fields

NeRF-MAE: Masked AutoEncoders for Self-Supervised 3D Representation Learning for Neural Radiance Fields

Muhammad Zubair Irshad, Sergey Zakharov, Vitor Guizilini, Adrien Gaidon, Zsolt Kira, Rares Ambrus

ECCV 2024 · 3D vision › NeRF and radiance fields · approach: Dynamic and editable NeRF · 20 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
pretrain NeRF-MAE encoder before fine-tuning for 3D OBB detection (ablation)3D Swin encoder + RPN head · ScanNet 3D OBBAP@50: 14.0 -> 17.0Improved
pretrain NeRF-MAE encoder for semantic voxel labelling (ablation)3D Swin encoder · Front3D segmentationmIOU: 24.9 -> 34.5Improved
use NeRF-MAE pretraining versus point-based pretraining baselines on NeRF gridFront3D voxel labelling · Front3D segmentationmIOU gain from pretraining: +9.6 vs +3.6 (PointM2AE)Improved
add more pretraining scenes (151 to 1515) (ablation)NeRF-MAE · Front3D 3D OBBAP@50: 0.45 -> 0.54Improved
add augmentation during pretraining (ablation)NeRF-MAE (Front3D only) · Front3D 3D OBBAP@50: 0.543 -> 0.591Improved
add HM3D and Hypersim to pretraining data (ablation)NeRF-MAE · ScanNet 3D OBBAP@50: 0.155 -> 0.170Improved
use higher-quality NeRFs for pretraining (18 vs 28 PSNR) (ablation)NeRF-MAE · Front3D 3D OBBAP@25: 36% AP25 improvementImproved
add skip connections in the decoder (ablation)NeRF-MAE · Front3D and ScanNet 3D OBBAP@25: 0.83 -> 0.85 (Front3D); 0.54 -> 0.57 (ScanNet)Improved
masking ratio 75% versus 65% or 85% (ablation)NeRF-MAE · 3D OBBAP@50: 0.519 / 0.512 -> 0.546Improved
pretrain on NeRF grid, fine-tune with fewer labelled scenesNeRF-MAE · Front3D 3D OBBAP@50: 0.42 with 61 scenes vs 0.41 with 122 scenes (NeRF-RPN)Improved
NeRF-MAE versus ImVoxelNet on Front3D AP@25 and AP@50NeRF-MAE · Front3D 3D OBBAP@50: 0.664 -> 0.630Hurt
NeRF-MAE versus NeRF-RPN on voxel super-resolutionNeRF-MAE · Voxel super-resolution 256^3PSNR: 16.25 -> 17.27Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy