Research map / Architectures and efficiency / RNNs and normalization
PointBeV: A Sparse Approach for BeV Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PointBeV, a sparse BeV segmentation model that predicts occupancy only at sampled BeV points using coarse/fine two-pass training, sparse feature pulling, and submanifold temporal attention, reaching SOTA on nuScenes vehicle, pedestrian and lane segmentation.
Problem
Dense BeV grids with fixed resolution and range allocate compute uniformly across all cells, making them memory-inefficient, especially for long temporal contexts and memory-constrained platforms.
Approach
Operate on sparse BeV points instead of dense grids: sample points, pull image features only from cameras where each lifted pillar point is visible, train with a coarse pass for exploration and a fine pass on densified anchor neighborhoods, and aggregate past frames with windowed submanifold attention over retained sparse points.
Key results
- PointBeV (EN-b4) reaches 38.7 vehicle IoU at 224x480 without visibility filtering vs 36.9 for Simple-BEV, and 44.0 vs 43.0 with visibility filtering.
- PointBeV reaches 18.5 pedestrian IoU vs 17.2 for FIERY and 49.6 lane IoU vs 44.8 for MatrixVT/PETRv2 on nuScenes.
- Training with only 20% of BeV points (8k) yields 38.3 vehicle IoU vs 38.3 with the full 40k points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PointBeV instead of Simple-BEV | vehicle segmentation on nuScenes · nuScenes val, no visibility filtering, 224 × 480 | IoU: 36.9∗ -> 38.7 | Helped |
| Use PointBeV-T instead of FIERY | temporal vehicle segmentation on nuScenes · nuScenes val, no visibility filtering, 224 × 480 | IoU: 38.2 -> 39.9 | Helped |
| Use PointBeV instead of SimpleBEV | vehicle segmentation on Lyft L5 · Lyft L5 Long | IoU: 44.5 -> 45.4 | Helped |
| Use PointBeV instead of TBP-Former static | pedestrian segmentation on nuScenes · nuScenes val, static | IoU: +1.3 IoU points | Helped |
| Use PointBeV instead of TBP-Former | pedestrian segmentation on nuScenes · nuScenes val, temporal | IoU: 18.6 -> 19.9 | Helped |
| Use PointBeV instead of MatrixVT | lane segmentation on nuScenes · nuScenes val, static | IoU: +4.8 IoU points | Helped |
| Add Sparse Feature Pulling (ablation) | PointBeV · nuScenes val, vehicle, batch size 1 | memory usage: 4.57 -> 3.50 | Helped |
| Add Sparse Feature Pulling (ablation) | PointBeV · nuScenes val, vehicle, batch size 1 | FPS: 13.70 -> 14.84 | Helped |
| Replace Submanifold Attention with standard attention (ablation) | PointBeV-T · nuScenes val, vehicle, temporal | IoU: 39.93 -> 40.49 | Mixed |
| Use two coarse/fine passes instead of only a coarse pass (ablation) | PointBeV training · nuScenes vehicle segmentation | IoU: 35.4 -> 38.3 | Helped |
| Reduce the training point budget from 100% to 20% (ablation) | PointBeV · nuScenes vehicle segmentation | IoU: 38.3 -> 38.3 | No clear effect |
Surprising findings
- Training with only 20% of the BeV points matches the performance of dense training.
- Submanifold temporal attention gives only marginal gains over standard attention while using about 5x more points.
Limitations
- Sparse inference requires simple dense completion by setting unsampled cells to empty, which may limit accuracy.
- Submanifold attention is computationally heavier, considering about 5x more points for marginal gains.
- Evaluated mainly on nuScenes and Lyft L5 with camera inputs, focusing on segmentation tasks.
Experimental setup
Datasets: nuScenes Lyft L5
Benchmarks: nuScenes Lyft L5
Metrics: IoU memory footprint (GiB) FPS
Baselines: FIERY CVT LaRa BEVFormer Simple-BEV BAEFormer TBP-Former MatrixVT PETRv2 M2BEV LSS ST-P3
Builds on: FIERY Simple-BEV BEVFormer NeRF hierarchical sampling submanifold sparse convolution attention
Closest papers by meaning
- DiffBEV: Conditional Diffusion Model for Bird’s Eye View Perception (AAAI 2024 · similarity 0.86)
- Improving Bird's Eye View Semantic Segmentation by Task Decomposition (CVPR 2024 · similarity 0.86)
- GeoBEV: Learning Geometric BEV Representation for Multi-view 3D Object Detection (AAAI 2025 · similarity 0.86)
- BEVDiffuser: Plug-and-Play Diffusion Model for BEV Denoising with Ground-Truth Guidance (CVPR 2025 · similarity 0.86)
- PC-BEV: An Efficient Polar-Cartesian BEV Fusion Framework for LiDAR Semantic Segmentation (AAAI 2025 · similarity 0.85)
- SparseAlign: a Fully Sparse Framework for Cooperative Object Detection (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.