Research map / Point clouds / Segmentation and detection
PTT: Point-Trajectory Transformer for Efficient Temporal 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTT, a point-trajectory transformer that fuses current-frame object points with long multi-frame proposal trajectories (long/short-term and future-aware encoding) for memory-efficient temporal LiDAR 3D detection on Waymo.
Problem
Temporal LiDAR 3D object detection with two-stage proposal-based methods requires storing per-frame object or whole point clouds, causing memory bank overhead, and combines point and trajectory features only by concatenation.
Approach
Store only current-frame object points plus multi-frame proposal trajectories (not per-frame point clouds), and fuse them with a transformer that models long-term, short-term, and predicted future trajectory memories, plus a point-trajectory aggregator with cross-attention.
Key results
- PTT with 64 frames reaches 81.32 L1 and 75.71 L2 mAPH on the Waymo validation set, versus 80.65/75.46 for MSF with 8 frames.
- Compared with FSD, PTT (64 frames) improves overall L2 mAPH on vehicles, pedestrians and cyclists by 5.6%, 6.9% and 2.0%.
- PTT with 64 frames runs in about 150 ms versus about 1100 ms for MPPNet with 16 frames, with lower memory bank size per the paper's analysis.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PTT with 64 frames instead of MSF with 8 frames | Waymo Open Dataset validation · Waymo Open Dataset val, ALL L2 | 3D mAPH: 75.46 -> 75.71 | Helped |
| add both long-term and short-term memory encoders (ablation) | PTT · Waymo Open Dataset val, L1 and L2 | mAP/mAPH: 79.61/78.22 -> 81.38/80.04; 74.21/72.79 -> 75.88/74.59 | Helped |
| add future encoding (ablation) | PTT · Waymo Open Dataset val, Vehicle, Pedestrian, Cyclist L1/L2 | APH: 81.31 / 74.00 -> 81.50 / 74.32; 81.80 / 74.73 -> 82.25 / 75.22; 75.98 / 73.82 -> 76.38 / 74.24 | Helped |
| add point-to-proposal and proposal-to-proposal features (ablation) | PTT · Waymo Open Dataset val, L1 and L2 | mAP/mAPH: 79.99/78.58 -> 81.38/80.04; 74.34/72.98 -> 75.88/74.59 | Helped |
| replace the MLP trajectory encoder with the point-trajectory transformer (ablation) | PTT · Waymo Open Dataset val, L2 Vehicle, Pedestrian, Cyclist | AP/APH: 72.22/71.74 -> 74.83/74.32; 76.51/73.65 -> 77.93/75.22; 74.19/73.46 -> 74.89/74.24 | Helped |
| use both self-attention and cross-attention in the point-trajectory aggregator (ablation) | PTT · Waymo Open Dataset val, L1 and L2 | mAP/mAPH: 80.00/78.58 -> 81.38/80.04; 74.31/72.95 -> 75.88/74.59 | Helped |
| increase the input sequence from 32 to 128 frames (ablation) | PTT · Waymo Open Dataset val, L1 and L2 | mAP/mAPH: 81.38/80.04 -> 81.46/80.11; 75.88/74.59 -> 75.95/74.68 | Hurt |
Surprising findings
- Performance slightly declines when using 128 past frames versus 64, which the authors attribute to redundant information.
Limitations
- Memory analysis is theoretical (space complexity comparison) rather than measured memory usage.
- Ablations use a light-cost setting (32 frames, 3 epochs) rather than the full configuration.
Experimental setup
Datasets: Waymo Open Dataset
Benchmarks: Waymo Open Dataset validation set
Metrics: 3D AP APH mAP mAPH
Baselines: MPPNet MSF CenterFormer FSD SWFormer PV-RCNN++ PillarNet SST 3D-MAN MoDAR
Builds on: MPPNet MSF CenterPoint CT3D PointRCNN
Closest papers by meaning
- MGTANet: Encoding Sequential LiDAR Points Using Long Short-Term Motion-Guided Temporal Attention for 3D Object Detection (AAAI 2023 · similarity 0.92)
- MAD: Memory-Augmented Detection of 3D Objects (CVPR 2025 · similarity 0.88)
- Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection (AAAI 2024 · similarity 0.88)
- iDet3D: Towards Efficient Interactive Object Detection for LiDAR Point Clouds (AAAI 2024 · similarity 0.87)
- Point Transformer V3: Simpler Faster Stronger (CVPR 2024 · similarity 0.87)
- FASTer: Focal token Acquiring-and-Scaling Transformer for Long-term 3D Objection Detection (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.