Research map / Detection and segmentation / 3D detection and driving
MAD: Memory-Augmented Detection of 3D Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a plug-and-play transformer module (MAD) that fuses a frozen 3D detector's proposals with long-term memory of past predictions, aligned via trajectory forecasts, and trains it with a progressive chunk-length schedule plus a memory-bank cache.
Problem
Most 3D object detectors only use a short temporal window (0.1-0.3s) of sensor data, so heavily occluded or distant objects are missed; long-term temporal fusion is computationally hard and prior object-based methods struggle with association and false negatives.
Approach
A sensor-agnostic add-on that stores timestamped refined proposals with trajectory forecasts in a memory bank, aligns them to the current time and ego pose by interpolation/extrapolation, learns to rescore and merge them with detector proposals, and refines them with memory cross-attention and factorized self-attention, trained with a schedule that grows chunk length and uses a cache of memory ban
Key results
- Adding MAD to SAFDNet 4f raises WOD validation Overall L2 APH from 77.1 to 79.4 (AP 78.4 to 81.0).
- MAD with SAFDNet 4f reaches 81.8 AP / 80.2 APH L2 on the WOD test leaderboard, best among online methods without ensembles or TTA.
- On AV2, MAD improves FCOS3D overall AP from 37.6 to 43.6 and BEVMap from 51.5 to 53.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MAD to CenterPoint 1f | CenterPoint 1f · WOD validation, Overall L1 | AP: 76.1 -> 82.9 | Helped |
| add MAD to FCOS3D | FCOS3D · AV2, vehicle in camera field of view | AP: 37.6 -> 43.6 | Helped |
| add memory proposals (ablation) | MAD on SAFDNet 4f · WOD validation, vehicle L2 | AP: 75.8 -> 77.0 | Helped |
| add memory attention (ablation) | MAD on SAFDNet 4f · WOD validation, vehicle L2 | AP: 76.5 -> 77.0 | Helped |
| add trajectory forecasting to align memory proposals (ablation) | MAD on SAFDNet 4f · WOD validation, vehicle L2 | AP: 76.9 -> 77.0 | Helped |
| add learned proposal rescoring (ablation) | MAD on SAFDNet 4f · WOD validation, vehicle L2 | AP: 72.7 -> 77.0 | Helped |
| add memory cache to single-frame chunk training (ablation) | MAD on SAFDNet 4f · WOD validation | AP L1: 83.9 -> 85.0 | Helped |
| add memory cache to 144-frame chunk training (ablation) | MAD on SAFDNet 4f · WOD validation | AP L1: 85.4 -> 85.2 | Hurt |
| use progressively longer chunks with memory cache (ablation) | MAD on SAFDNet 4f · WOD validation | AP L1: 84.9 -> 85.8 | Helped |
Surprising findings
- Naively taking the union of detection and memory proposals performs far worse than the base detector (APH L2 16.4 vs 77.1) due to miscalibrated scores, so learned rescoring is essential.
- MAD-augmented single-frame detectors outperform multi-frame detectors, suggesting memory fusion beats point aggregation.
Limitations
- Requires a separate MAD model trained for each base detector.
- Relies on the ego pose being localized and trajectory forecasts that grow uncertain with horizon.
- Detector must be frozen and pre-trained; WOD license prevents reusing pre-trained detectors, so baselines were retrained.
Experimental setup
Datasets: Waymo Open Dataset Argoverse 2 Sensor dataset
Benchmarks: Waymo Open Dataset 3D detection Argoverse 2 Sensor camera 3D detection
Metrics: AP APH mAP
Baselines: CenterPoint HEDNet SAFDNet FCOS3D BEVMap MPPNet MSF PTT LEF MoDAR
Builds on: DeTra DETR CenterPoint SAFDNet HEDNet
Closest papers by meaning
- PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection (CVPR 2024 · similarity 0.88)
- MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection (ICLR 2025 · similarity 0.87)
- Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection (AAAI 2024 · similarity 0.86)
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.86)
- PromptDet: A Lightweight 3D Object Detection Framework with LiDAR Prompts (AAAI 2025 · similarity 0.86)
- TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perception (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.