Research map / Architectures and efficiency / Attention and transformers
FASTer: Focal token Acquiring-and-Scaling Transformer for Long-term 3D Objection Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FASTer, a lidar multi-frame 3D detector that adaptively selects focal tokens via Ad-MHSA scaling and fuses frames with Grouped Hierarchical Fusion, achieving SOTA on Waymo Open Dataset with lower latency and memory.
Problem
Region-based temporal 3D lidar detectors sample a fixed large number of points per proposal and concatenate per-frame results, so storage and compute grow exponentially with the number of frames and global cross-frame information is poorly exchanged.
Approach
Adaptive Scaling (Ad-MHSA) scores each point token by its attention-based contribution to the global representation and keeps only top-scoring focal tokens; only focal points are stored in a memory bank, and historical frames are fused via grouped hierarchical fusion with intra-group max-pool fusion rather than result-level concatenation.
Key results
- 16-frame FASTer reaches 81.49 mAPH (L1) and 75.92 mAPH (L2) on the Waymo validation set, above MPPNet (16-frame) at 80.40/74.85 and PTT (64-frame) at 81.32/75.71.
- On the Waymo test set, 32-frame FASTer reaches 83.55 mAP / 82.23 mAPH (L1) and 78.82 / 77.54 (L2), above MSF (8-frame) at 83.11/81.74 (L1).
- 16-frame FASTer has 75 ms delay and 2856M memory on an NVIDIA 3090, versus 400 ms and 6083M for MSF with 8 frames, and 99 ms and 6938M for PTT with 32 frames.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FASTer instead of PTT | Waymo Open Dataset validation, 16-frame detection · Waymo Open Dataset validation | mAPH L2: 75.71 -> 75.92 | Helped |
| use FASTer instead of MSF | Waymo Open Dataset test, 16-frame detection · Waymo Open Dataset test | mAPH L2: 76.96 -> 77.21 | Helped |
| use FASTer instead of MSF | Waymo Open Dataset, region-based detection · Waymo Open Dataset | latency: 101ms -> 75 ms | Helped |
| add the SSP decoder layer (ablation) | FASTer · Waymo Open Dataset validation, 16 frames | mAPH L2: 75.51 -> 75.92 | Helped |
| add the MSP decoder layer (ablation) | FASTer · Waymo Open Dataset validation, 16 frames | mAPH L2: 73.19 -> 75.92 | Helped |
| add Intra-Group Fusion (ablation) | FASTer · Waymo Open Dataset validation, 16 frames | mAPH L2: 75.45 -> 75.92 | Helped |
| add motion feature encoding (ablation) | FASTer · Waymo Open Dataset validation, 16 frames | mAPH L2: 74.72 -> 75.92 | Helped |
| replace spatiotemporal fusion with separate decoding and concatenation (ablation) | FASTer · Waymo Open Dataset validation, Level 2 | APH: 75.57 -> 75.58 | Hurt |
| replace spatiotemporal fusion with Cross Attention (ablation) | FASTer · Waymo Open Dataset validation, Level 2 | APH: 75.22 -> 75.58 | Hurt |
| replace spatiotemporal fusion with BiFA (ablation) | FASTer · Waymo Open Dataset validation, Level 2 | APH: 75.01 -> 75.58 | Hurt |
| replace adaptive scaling with supervised scaling (ablation) | FASTer · Waymo Open Dataset validation | Vehicle APH L2: 74.85 -> 75.57 | Hurt |
| replace adaptive scaling with attention-mask scaling (ablation) | FASTer · Waymo Open Dataset validation | Vehicle APH L2: 74.60 -> 75.57 | Hurt |
Surprising findings
- Adaptive Scaling outperforms in-box supervision and attention masking, which the authors attribute to those methods focusing on local rather than global tokens.
Limitations
- Relies on a 4-frame CenterPoint RPN and proposal trajectories, so proposal recall still matters.
- Requires a staged training strategy because focal points cannot be obtained in advance.
Experimental setup
Datasets: Waymo Open Dataset
Benchmarks: Waymo Open Dataset
Metrics: mAP mAPH latency (ms) GPU memory (M)
Baselines: MPPNet MSF PTT ScatterFormer CenterFormer PillarNeXt-B VoxelNext DSVT SST PV-RCNN
Builds on: MPPNet MSF PTT CenterPoint CT3D
Closest papers by meaning
- PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection (CVPR 2024 · similarity 0.87)
- MAD: Memory-Augmented Detection of 3D Objects (CVPR 2025 · similarity 0.85)
- MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection (ICLR 2025 · similarity 0.85)
- SwiftPillars: High-Efficiency Pillar Encoder for Lidar-Based 3D Detection (AAAI 2024 · similarity 0.85)
- Good Token Hunting: A Hitchhiker's Guide to Token Selection for Visual Geometry Transformers (NeurIPS 2026 · similarity 0.85)
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.