Research map / Detection and segmentation / 3D detection and driving
MOS: Model Synergy for Test-Time Adaptation on LiDAR-Based 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Model Synergy (MOS), an online test-time adaptation method for LiDAR 3D detectors that assembles a bank of historical checkpoints via synergy weights derived from an inverse generalized Gram matrix over box and feature similarities.
Problem
Test-time adaptation of LiDAR-based 3D object detectors under cross-dataset shifts, sensor/weather corruptions, and their combination (cross-corruption), where existing TTA methods designed for classification or 2D detection perform poorly and multi-epoch UDA is impractical.
Approach
Instead of averaging all past checkpoints uniformly as in mean-teacher methods, dynamically select and weight a bank of K historical checkpoints per test batch via synergy weights w = G^{-1} 1, where the Gram matrix uses output-level box-set Hungarian similarity and feature-level rank-based similarity; the weighted super model generates pseudo labels for a single-iteration update, and the bank dro
Key results
- On Waymo to KITTI, MOS reaches 64.16 AP3D (moderate) vs. 35.34 for the best TTA baseline CoTTA and 61.83 for UDA method ST3D.
- On cross-corruption (Waymo to KITTI-C), MOS surpasses the highest baseline and direct inference by 67.3% and 161.5% respectively, across corruptions.
- Keeping 3 SW-selected checkpoints outperforms assembling the latest 20 checkpoints by 18.42% AP3D while reducing memory by 85%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply MOS instead of ST3D | SECOND · Waymo → KITTI | AP3D moderate: 61.83 -> 64.16 | Helped |
| apply MOS instead of ST3D | SECOND · nuScenes → KITTI | AP3D moderate: 54.13 -> 51.11 | Hurt |
| apply MOS instead of CoTTA | SECOND · KITTI → KITTI-C, heavy corruption mean | AP3D moderate: 58.71 -> 61.82 | Helped |
| apply MOS instead of ST3D | SECOND · Waymo → nuScenes | AP3D moderate: 20.19 -> 18.75 | Hurt |
| add model assembly using weighted checkpoint averaging (ablation) | MOS on SECOND · Waymo → KITTI | AP3D moderate: 43.14 -> 64.16 | Helped |
| replace mean averaging with MOS weighted checkpoint assembly (ablation) | MOS on SECOND · Waymo → KITTI | AP3D moderate: 45.89 -> 64.16 | Helped |
| add feature-level similarity to the synergy weights (ablation) | MOS on SECOND · Waymo → KITTI | AP3D moderate: 59.98 -> 64.16 | Helped |
| add output-level box similarity to the synergy weights (ablation) | MOS on SECOND · Waymo → KITTI | AP3D moderate: 61.40 -> 64.16 | Helped |
| add feature-level similarity to the synergy weights (ablation) | MOS on SECOND · Waymo → KITTI-C, incomplete echo | AP3D hard: 11.93 -> 14.96 | Helped |
| add output-level box similarity to the synergy weights (ablation) | MOS on SECOND · Waymo → KITTI-C, incomplete echo | AP3D hard: 12.38 -> 14.96 | Helped |
| replace cosine feature similarity with rank-based feature similarity (ablation) | MOS on SECOND · Waymo → KITTI-C, incomplete echo | AP3D moderate: 12.51 -> 16.14 | Helped |
| use BEV feature maps instead of RoI features for feature similarity (ablation) | MOS on SECOND · Waymo → KITTI-C, incomplete echo | AP3D hard: 11.84 -> 14.96 | Helped |
Surprising findings
- Assembling the latest 20 checkpoints yields only marginal gains over the latest 5, suggesting recent checkpoints converge and lose unique knowledge.
- Cosine similarity for feature-level comparison underperforms the rank-based measure by about 29% in moderate AP3D, and gives no gain over removing the feature term.
Limitations
- High computational demand and memory cost from storing and assembling full checkpoints.
- Pseudo-labeling struggles with small, rare objects such as pedestrians under severe corruptions.
- Results reported for car class in main text; multi-class results are limited to a subset of tasks.
Experimental setup
Datasets: KITTI Waymo nuScenes KITTI-C
Benchmarks: KITTI nuScenes KITTI-C Waymo
Metrics: AP_3D AP_BEV Recall@0.5 closed gap
Baselines: No Adapt. SN ST3D Tent CoTTA SAR MemCLR Oracle
Builds on: Tent CoTTA ST3D MemCLR OpenPCDet
Closest papers by meaning
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.90)
- FIND: Frequency Invariance Disentanglement for Test-Time Adaptation in LiDAR 3D Detection (NeurIPS 2026 · similarity 0.89)
- D^3CTTA: Domain-Dependent Decorrelation for Continual Test-Time Adaption of 3D LiDAR Segmentation (CVPR 2025 · similarity 0.88)
- GBlobs: Explicit Local Structure via Gaussian Blobs for Improved Cross-Domain LiDAR-based 3D Object Detection (CVPR 2025 · similarity 0.87)
- CMDA: Cross-Modal and Domain Adversarial Adaptation for LiDAR-Based 3D Object Detection (AAAI 2024 · similarity 0.87)
- MAD: Memory-Augmented Detection of 3D Objects (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.