Research map / Detection and segmentation / 3D detection and driving
Real-time Stereo-based 3D Object Detection for Streaming Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StreamDSGN, a real-time stereo-based 3D detector for streaming perception that predicts next-frame 3D boxes using feature-flow fusion, a motion consistency loss, and a large kernel BEV backbone, evaluated on KITTI Tracking.
Problem
Stereo-based 3D object detectors are too slow for streaming perception, where latency causes misalignment between predictions and the scene; features at the current frame do not match the next-frame ground truth used in streaming evaluation.
Approach
Predict the next frame's 3D boxes from current and historical BEV features; warp the current feature to the next-frame grid via similarity-based feature flow (pseudo-next feature); add explicit velocity and acceleration regression supervision from historical ground-truth trajectories; and use a large-kernel (VAN-style) BEV backbone for long-range context.
Key results
- StreamDSGN reaches 77.47/63.76/57.42 sAP3D (Easy/Mod/Hard) for Car at IoU 0.7 with 91.45ms latency, versus 73.02/58.37/51.86 for StreamYOLO on DSGN++_opt.
- Ablation: the full model improves sAP3D at the hard level by 4.33% over the streaming baseline (53.11 to 57.42).
- Real-time optimization reduces DSGN++ latency from 263.33ms to 80.71ms with a small offline accuracy loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FFF, MCL, and LKBB (ablation) | Historical-feature baseline · KITTI Tracking Car, Hard | sAP 3D: 53.11 -> 57.42; 4.33% increase | Helped |
| Add FFF (ablation) | Historical-feature baseline · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 75.38 -> 75.88; 59.55 -> 61.57; 53.11 -> 54.77 | Helped |
| Add MCL (ablation) | Historical-feature baseline · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 75.38 -> 78.15; 59.55 -> 62.28; 53.11 -> 56.80 | Helped |
| Replace the hourglass backbone with LKBB (ablation) | Historical-feature baseline · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 75.38 -> 76.74; 59.55 -> 60.31; 53.11 -> 55.01 | Helped |
| Predict the next frame with a real-time detector instead of detecting the current frame (ablation) | DSGN++opt · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 25.50 -> 38.62; 17.68 -> 32.55; 14.75 -> 29.39 | Helped |
| Fuse the previous-frame feature (ablation) | Next-frame prediction baseline · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 38.62 -> 75.38; 32.55 -> 59.55; 29.39 -> 53.11 | Helped |
| Fuse historical features before the BEV backbone rather than before the detection head (ablation) | StreamYOLO-style pipeline · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 73.02 -> 75.38; 58.37 -> 59.55; 51.86 -> 53.11 | Helped |
| Use FFF with downsample ratio 2 and maximum displacement 4 instead of 3 (ablation) | FFF · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 75.88 -> 76.82; 61.57 -> 60.73; 54.77 -> 54.02 | Mixed |
| Use FFF with downsample ratio 4 instead of 2 (ablation) | FFF · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 75.88 -> 76.11; 61.57 -> 59.40; 54.77 -> 53.82 | Mixed |
| Set MCL acceleration-loss weight to 0.8 instead of 0.0 (ablation) | MCL · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 76.13 -> 78.15; 61.26 -> 62.28; 54.40 -> 56.80 | Helped |
| Set MCL acceleration-loss weight to 1.0 instead of 0.8 (ablation) | MCL · KITTI Tracking Car, sAP 3D Easy, Moderate, and Hard | sAP 3D: 78.15 -> 75.74; 62.28 -> 59.62; 56.80 -> 54.24 | Hurt |
Surprising findings
- Fusing historical features before the BEV backbone outperforms fusing before the detection head (as in StreamYOLO), and the authors attribute this to the detection head's shallow receptive field.
Limitations
- FFF can produce incorrect pseudo-next features for occluded or truncated objects due to erroneous similarity matching.
- Validated only on stereo-based methods, not multi-view camera systems.
- Evaluated only on KITTI Tracking with a custom split; no error bars reported.
Experimental setup
Datasets: KITTI Tracking
Benchmarks: KITTI Tracking
Metrics: streaming average precision (sAP) BEV sAP 3D latency (ms)
Baselines: DSGN++ Streamer (DSGN++) Streamer (DSGN++_opt) StreamYOLO
Builds on: DSGN++ StreamYOLO Visual Attention Network Streamer
Closest papers by meaning
- Real-Time Object Detection for Streaming Perception (CVPR 2022 · similarity 0.90)
- Are We Ready for Vision-Centric Driving Streaming Perception? The ASAP Benchmark (CVPR 2023 · similarity 0.88)
- Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception (AAAI 2025 · similarity 0.86)
- OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection (NeurIPS 2025 · similarity 0.86)
- CorrBEV: Multi-View 3D Object Detection by Correlation Learning with Multi-modal Prototypes (CVPR 2025 · similarity 0.85)
- StreamForest: Efficient Online Video Understanding with Persistent Event Memory (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.