Research map / Point clouds / Segmentation and detection

Multi-Scale Neighborhood Occupancy Masked Autoencoder for Self-Supervised Learning in LiDAR Point Clouds

CVPR 2025 · Point clouds › Segmentation and detection · approach: LiDAR voxel and pillar detectors · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NOMAE, a multi-scale sparse masked occupancy autoencoder for LiDAR point clouds that reconstructs occupancy only in the neighborhood of visible voxels, with hierarchical mask generation, and reports state-of-the-art nuScenes and Waymo segmentation and nuScenes detection results.

Problem

Masked autoencoder pretraining on automotive LiDAR point clouds is hard because most of the 3D volume is empty, which leaks occupancy information to the decoder and makes full-volume reconstruction computationally expensive, limiting SSL mostly to 2D BEV encoders.

Approach

Reconstruct occupancy only for voxels in a local neighborhood of visible voxels, apply a separate sparse decoder at each feature scale (multi-scale pretext), and generate consistent multi-scale masks with a hierarchical mask generator so coarse-scale masking ratios stay high without shrinking fine-scale neighborhoods.

Key results

• NOMAE with PTv3 reaches 81.8 mIoU on nuScenes val semantic segmentation after fine-tuning, +1.4 over PTv3 baseline (80.4).
• On nuScenes object detection with 20% labels, NOMAE achieves 60.9 NDS and 54.4 mAP, improving over UVTR-L baseline (46.7 NDS, 39.0 mAP) and UniPAD (55.8 NDS, 48.1 mAP).
• NOMAE sets 70.3 mIoU on the Waymo single-frame semantic segmentation test set, a new state of the art at submission time.

Measured effects reported in the paper

Surprising findings

• Reconstructing only a local neighborhood of visible voxels outperforms reconstructing the whole scene (Occupancy-MAE style), and dropping far-away isolated masked points from the loss improves performance.

Limitations

• Sensitive to LiDAR point density; not evaluated on sparser sensors such as radar.
• Does not exploit the temporal nature of LiDAR sequences.
• High-resolution sparse 3D encoders are less explored for object detection, where BEV and low-resolution 3D methods still dominate.

Experimental setup

Datasets: nuScenes Waymo Open Dataset

Benchmarks: nuScenes Waymo Open Dataset

Metrics: mIoU mAcc fwIoU NDS mAP

Baselines: Occupancy-MAE GeoMAE UniPAD GD-MAE ALSO Learning from 2D PTv3 UVTR-L MinkUNet SphereFormer PTv2

Builds on: Occupancy-MAE Point Transformer V3 GeoMAE Point-M2AE UVTR

Closest papers by meaning

• 3D Feature Prediction for Masked-AutoEncoder-Based Point Cloud Pretraining (ICLR 2024 · similarity 0.88)
• BEV-MAE: Bird’s Eye View Masked Autoencoders for Point Cloud Pre-training in Autonomous Driving Scenarios (AAAI 2024 · similarity 0.88)
• Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders (AAAI 2024 · similarity 0.88)
• Learning Occupancy for Monocular 3D Object Detection (CVPR 2024 · similarity 0.87)
• UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.87)
• VLM2Scene: Self-Supervised Image-Text-LiDAR Learning with Foundation Models for Autonomous Driving Scene Understanding (AAAI 2024 · similarity 0.86)