Research map / Point clouds / Segmentation and detection
TASeg: Temporal Aggregation Network for LiDAR Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TASeg, a multi-frame LiDAR segmentation network that aggregates long-term temporal point clouds with class-specific steps, distills from a gt-prior teacher, fuses temporal camera images, and augments static/moving samples, reaching 1st on three SemanticKITTI/nuScenes tracks.
Problem
LiDAR semantic segmentation is limited by point cloud sparsity, and existing multi-frame methods can only use a few frames due to GPU memory, while ignoring temporal camera images and struggling with static vs. moving distinction in multi-scan settings.
Approach
Assign class-dependent temporal aggregation steps using historical predictions (Flexible Step Aggregation), distill from a teacher given ground-truth group masks (Mask Distillation), aggregate temporal image features via LiDAR points as a bridge for fusion, and switch objects' motion states via point-cloud manipulation for augmentation.
Key results
- TASeg achieves 76.5 mIoU on the SemanticKITTI single-scan test set, 1.3 above UniSeg, ranking 1st.
- On the SemanticKITTI multi-scan test set, TASeg reaches 65.7 mIoU, 3.3 above 2DPASS.
- FSA with 71.3 mIoU on the validation set uses 8.5G memory versus 69.7G for MSeg3D with 69.9 mIoU; mask distillation adds 0.5 mIoU to reach 71.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TASeg instead of UniSeg | SemanticKITTI single-scan segmentation · SemanticKITTI test, single-scan | mIoU: 75.2 -> 76.5 | Helped |
| Use TASeg instead of 2DPASS | SemanticKITTI multi-scan segmentation · SemanticKITTI test, multi-scan | mIoU: 62.4 -> 65.7 | Helped |
| Use TASeg instead of UniSeg | nuScenes LiDAR segmentation · nuScenes test | mIoU: 83.5 -> 84.6 | Helped |
| Add Flexible Step Aggregation (ablation) | MinkowskiNet on SemanticKITTI · SemanticKITTI val | mIoU: 68.9 -> 71.3 | Helped |
| Replace historical-prediction masks with ground-truth masks for FSA (ablation) | FSA on SemanticKITTI · SemanticKITTI val | mIoU: 71.3 -> 75.9 | Helped |
| Add Mask Distillation to FSA (ablation) | FSA on SemanticKITTI · SemanticKITTI val | mIoU: 71.3 -> 71.8 | Helped |
| Reduce model parameters to 75% (ablation) | FSA + Mask Distillation · SemanticKITTI val | mIoU: 71.8 -> 71.6 | Hurt |
| Increase the FSA temporal window from 4 to 24 (ablation) | FSA · SemanticKITTI val | mIoU: 69.9 -> 71.6 | Helped |
| Use FSA group division 3 instead of division 1 (ablation) | FSA · SemanticKITTI val | mIoU: 70.8 -> 71.3 | Helped |
| Add temporal images to TIAF, increasing from 0 to 7 images (ablation) | TLAD on SemanticKITTI · SemanticKITTI val | mIoU: 71.8 -> 72.8 | Helped |
| Add Static-Moving Switch Augmentation (ablation) | TASeg on SemanticKITTI multi-scan · SemanticKITTI test, multi-scan | mIoU: 61.3 -> 65.7 | Helped |
| Apply TASeg to Cylinder3D (ablation) | Cylinder3D · SemanticKITTI val | mIoU: 66.0 -> 69.6 | Helped |
Surprising findings
- Using ground-truth group masks for FSA boosts validation mIoU to 75.9, a much larger gain than the distilled student achieves.
Limitations
- Temporal image aggregation and TIAF rely on pose and calibration to project features, and the paper notes the approach is not the focus for complex fusion.
- Group division and step choices are hand-tuned based on per-class validation performance.
Experimental setup
Datasets: SemanticKITTI nuScenes
Benchmarks: SemanticKITTI nuScenes
Metrics: mIoU latency memory
Baselines: MSeg3D SVQNet 2DPASS UniSeg SphereFormer Cylinder3D SPVNAS RPVNet MarS3D TemporalLidarSeg
Builds on: MinkowskiNet 2DPASS MSeg3D SVQNet PCSeg
Closest papers by meaning
- PC-BEV: An Efficient Polar-Cartesian BEV Fusion Framework for LiDAR Semantic Segmentation (AAAI 2025 · similarity 0.87)
- TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perception (NeurIPS 2025 · similarity 0.86)
- HiLoTs: High-Low Temporal Sensitive Representation Learning for Semi-Supervised LiDAR Segmentation in Autonomous Driving (CVPR 2025 · similarity 0.85)
- LiFi: LiDAR Generation from Multi-View Images via Geometric and Semantic Collaborative Guidance (NeurIPS 2026 · similarity 0.85)
- LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes (CVPR 2025 · similarity 0.85)
- Multi-Space Alignments Towards Universal LiDAR Segmentation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.