Research map / Detection and segmentation / CNN features
Transtreaming: Adaptive Delay-aware Transformer for Real-time Streaming Perception
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Transtreaming, a transformer-based streaming perception detector (Transtreamer) with an adaptive scheduler that predicts multiple future frames and dispatches the output matching the present time, evaluated on Argoverse-HD under varying computational delays.
Problem
Real-time streaming object detection in autonomous driving where computational delays fluctuate with workload, so a detector's output lags the present and fixed single-frame forecasting breaks under high or bursty delay.
Approach
A temporal adaptive transformer with relative temporal positional embedding decodes features for any set of future frames from buffered past features, and an adaptive strategy plans which past and future frame indices to use based on estimated runtime delay, with feature and output buffers.
Key results
- Transtreaming-L reaches 38.2 sAP on Argoverse-HD, 0.4 above the previous best DAMO-StreamNet-L (37.8).
- Transtreaming-S* beats DAMO-StreamNet-S by about 1.1-2.8 sAP across four devices and delay factors d in {2,4,8,16}.
- Removing the planner and buffer drops sAP_1 by 25.8 points in the ablation, showing the buffer's importance for streaming efficiency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Transtreaming-L instead of DAMO-StreamNet-L | Argoverse-HD streaming perception · Argoverse-HD | sAP: 37.8 -> 38.2 | Helped |
| Use Transtreaming-S* instead of LongShortNet-S | Argoverse-HD streaming perception on v100 cluster · v100 cluster | sAP_2: 25.5 -> 26.4 | Helped |
| Use Transtreaming-S* instead of LongShortNet-S | Argoverse-HD streaming perception on 4080, 3090, and 2080Ti servers · Different real-world devices | sAP_2: 24.5 -> 25.3; 24.0 -> 24.9; 23.0 -> 23.5 | Helped |
| Use mixed speed training | Transtreaming-S under vehicle acceleration adaptation · Vehicle acceleration adaptation | mAP: 29.5 -> 28.9; 23.1 -> 24.5; 15.2 -> 17.9; 9.6 -> 12.3 | Mixed |
| Add the Planner (ablation) | Transtreaming-S with Buffer · Argoverse-HD streaming | sAP: 29.7 -> 30.1 | Helped |
| Add the Buffer (ablation) | Transtreaming-S without Planner or Buffer · Argoverse-HD streaming | sAP: 3.9 -> 29.7 | Helped |
| Add RTPE (ablation) | Transtreaming-S with TAT · Argoverse-HD | sAP: 29.9 -> 30.8 | Helped |
| Add TAT (ablation) | Transtreaming-S with RTPE · Argoverse-HD | sAP: 29.9 -> 30.8 | Helped |
| Replace RTPE with sinusoidal or learnable positional encoding (ablation) | Transtreaming · Argoverse-HD | test sAP | Hurt |
| Replace TAT with the baseline neck (ablation) | Transtreaming · Argoverse-HD | test sAP | Hurt |
Surprising findings
- LongShortNet, with a lighter backbone, outperforms DAMO-StreamNet in high-latency scenarios, suggesting larger models do not scale well with added latency.
Limitations
- Temporal proposals only enter via attention weights rather than interacting directly with input tokens.
- Computation delay adaptation metric can suffer from temporal aliasing when delay nearly aligns with frame-rate multiples.
Experimental setup
Datasets: Argoverse-HD COCO
Benchmarks: Argoverse-HD
Metrics: sAP sAP50 sAP75 sAP_S sAP_M sAP_L sAP_d (delay factor 2,4,8,16) mAP_2/4/8/16 (vehicle acceleration)
Baselines: StreamYOLO DAMO-StreamNet LongShortNet DADE YOLOX Adaptive Streamer
Builds on: StreamYOLO DAMO-StreamNet LongShortNet Adaptive Streamer
Closest papers by meaning
- Real-time Stereo-based 3D Object Detection for Streaming Perception (NeurIPS 2024 · similarity 0.86)
- DETRs Beat YOLOs on Real-time Object Detection (CVPR 2024 · similarity 0.83)
- DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving (ICLR 2025 · similarity 0.83)
- VISTREAM: Improving Computation Efficiency of Visual Streaming Perception via Law-of-Charge-Conservation Inspired Spiking Neural Network (CVPR 2025 · similarity 0.83)
- StreamForest: Efficient Online Video Understanding with Persistent Event Memory (NeurIPS 2025 · similarity 0.83)
- Ev-3DOD: Pushing the Temporal Boundaries of 3D Object Detection with Event Cameras (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.