Research map / Point clouds / Segmentation and detection
Just Add $100 More: Augmenting Pseudo-LiDAR Point Cloud for Resolving Class-imbalance Problem
Mincheol Chang, Siyeong Lee, Jinkyu Kim, Namil Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PGT-Aug, which turns low-cost surround-view videos of miniatures and public videos into NeRF/Gaussian-splat-based pseudo-LiDAR objects for minority-class augmentation in LiDAR 3D detection, with domain alignment and map-aware placement.
Problem
Class imbalance (long-tailed distribution) in LiDAR-based 3D object detection datasets, where minority classes have few samples and existing copy-paste augmentation has limited sample diversity and restricted placement.
Approach
Reconstruct volumetric 3D instances of minority-class objects from multi-view video (miniatures bought for about $100 or public videos) with Plenoxels/Gaussian Splatting, convert them to LiDAR-like pseudo point clouds via range-view rearrangement and a CycleGAN-based intensity estimator, and place them using ground-plus-map-aware insertion areas.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PGT-Aug pseudo-LiDAR samples with ground+map placement instead of Real-Aug | CP-Voxel · nuScenes val | mAP: 62.27 -> 63.52 | Improved |
| Add PGT-Aug pseudo-LiDAR samples with ground+map placement instead of Real-Aug | CP-Voxel · nuScenes val | NDS: 68.39 -> 69.11 | Improved |
| Replace Real-Aug with PGT-Aug | CP-Voxel · nuScenes test | mAP: 65.8 -> 67.1 | Improved |
| Replace Real-Aug with PGT-Aug | CP-Voxel · nuScenes test | NDS: 71.3 -> 72.3 | Improved |
| Replace GT-Aug with PGT-Aug | CP-Voxel · Lyft val | mAP: 13.84 -> 14.01 | Improved |
| Replace GT-Aug with PGT-Aug | SECOND · KITTI val | mAP: 68.5 -> 70.1 | Improved |
| Use azimuth resolution 3600 px instead of 1080 px (ablation) | Plenoxel pseudo-LiDAR generation · nuScenes val pseudo-LiDAR quality | Avg. FID Score: 7.7 -> 14.2 | Hurt |
| Add group intensity loss (ablation) | Plenoxel pseudo-LiDAR generation without RGB features · nuScenes val pseudo-LiDAR quality | Avg. FID Score: 8.2 -> 8.3 | Hurt |
| Add RGB features (ablation) | Plenoxel pseudo-LiDAR generation without group intensity loss · nuScenes val pseudo-LiDAR quality | Avg. FID Score: 8.2 -> 8.3 | Hurt |
| Replace Plenoxel with Gaussian Splatting (ablation) | PGT-Aug · nuScenes val pseudo-LiDAR quality | Avg. FID Score: 7.7 -> 7.1 | Improved |
| Add ray tracing (ablation) | PGT-Aug with CP-Voxel · nuScenes val | mAP: 63.5 -> 63.8 | Improved |
| Add ray tracing (ablation) | PGT-Aug with Transfusion-L · nuScenes val | mAP: 64.2 -> 65.0 | Improved |
Surprising findings
- Augmentation with generated pseudo-LiDAR objects improves minority classes without degrading majority-class accuracy, which the authors highlight as a confirmation that the samples are high quality.
- Pseudo-LiDAR objects generated from Gaussian Splatting reach FID 7.1 on nuScenes, lower than some real datasets (Lyft 4.8 is lower, but A2D2 14.4 is higher), suggesting the quality is comparable to real data.
Limitations
- A domain discrepancy remains between real and pseudo point clouds, requiring a mixing ratio of original and pseudo objects.
- Generation depends on surround-view videos or miniatures; collected objects are limited to a few classes (31 miniature and 97 public-video sequences).
Experimental setup
Datasets: nuScenes KITTI Lyft A2D2
Benchmarks: nuScenes KITTI Lyft
Metrics: AP mAP NDS FID
Baselines: GT-Aug Real-Aug CP-Voxel CP-Pillar SECOND Transfusion-L VoxelNeXt
Builds on: GT-Aug Real-Aug Plenoxels 3D Gaussian Splatting CycleGAN
Closest papers by meaning
- LiDAR-Aug: A General Rendering-Based Augmentation Framework for 3D Object Detection (CVPR 2021 · similarity 0.87)
- End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection (CVPR 2020 · similarity 0.86)
- LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection (ICLR 2024 · similarity 0.85)
- NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields (AAAI 2024 · similarity 0.84)
- Pre-training LiDAR-based 3D Object Detectors through Colorization (ICLR 2024 · similarity 0.84)
- LiDAR-Net: A Real-scanned 3D Point Cloud Dataset for Indoor Scenes (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
