Research map / Architectures and efficiency / Pruning and efficient tuning
Is Multiple Object Tracking a Matter of Specialization?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PASTA, a modular PEFT framework for query-based (MOTRv2-style) multi-object trackers, training attribute-specific LoRA and scale-shift experts on MOTSynth and merging them in parameter space for zero-shot transfer to MOT17 and PersonPath22.
Problem
End-to-end transformer trackers suffer from negative interference when trained on heterogeneous scenarios and from poor domain generalization, often requiring costly fine-tuning for new domains.
Approach
Define scene attributes (lighting, viewpoint, occupancy, location, camera motion), train an independent PEFT expert per attribute value (LoRA for linear layers, channel-wise scale-and-shift for ResNet convolutions), and compose the selected experts by weighted averaging of task vectors in parameter space, with soft routing (rho=0.8) for domain-shift settings.
Key results
- On MOTSynth test, PASTA (15M trainable params) reaches 53.0 HOTA and 57.6 IDF1 versus 52.4 HOTA and 56.6 IDF1 for MOTRv2-MS (42M params).
- Zero-shot synth-to-real on MOT17: PASTA (rho=0.8) achieves 64.0 HOTA and 74.9 IDF1 vs 62.6 HOTA and 73.0 IDF1 for MOTRv2-MS.
- Zero-shot on PersonPath22: PASTA (rho=0.8) reaches 50.0 MOTA and 53.8 IDF1 vs 48.3 MOTA and 53.1 IDF1 for MOTRv2-MS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use attribute-specialized modules instead of monolithic fine-tuning | MOTRv2-MS · MOTSynth test | HOTA: 52.4 -> 53.0 | Helped |
| use soft weighted aggregation (rho=0.8) instead of hard module selection (rho=1) (ablation) | PASTA · MOT17 zero-shot | HOTA: 63.7 -> 64.0 | Helped |
| use soft weighted aggregation (rho=0.8) instead of hard module selection (rho=1) (ablation) | PASTA · PersonPath22 test, MOTSynth-to-real zero-shot | MOTA: 49.7 -> 50.0 | Helped |
| average selected modules instead of summing them (ablation) | PASTA · MOTSynth validation | HOTA: 0.65 -> 60.1 | Helped |
| select Domain Expert modules instead of opposite scenario modules (ablation) | PASTA · MOTSynth validation | HOTA: 59.2 -> 60.1 | Helped |
| apply modules to the decoder instead of excluding it (ablation) | PASTA · MOTSynth test | HOTA: 51.5 -> 53.0 | Helped |
| apply modules to the encoder instead of excluding it (ablation) | PASTA · MOTSynth test | HOTA: 52.4 -> 53.0 | Helped |
| apply modules to the visual backbone instead of excluding it (ablation) | PASTA · MOTSynth test | HOTA: 52.5 -> 53.0 | Helped |
| use weighted aggregation of all modules instead of averaging selected modules only (ablation) | PASTA · PersonPath22 test, zero-shot | MOTA: 49.6 -> 50.0 | Helped |
| use attribute-specialized modules instead of full fine-tuning | MOTRv2-MS · MOTSynth after subsequent MOT17 training | HOTA: 48.1 -> 49.8 | Helped |
Surprising findings
- Summing selected modules instead of averaging produced near-zero results, which the authors attribute to weight-magnitude alteration.
- Using opposite (mismatched) modules still performs relatively well, suggesting modules assist one another via general domain knowledge.
Limitations
- Relies on an expert router requiring manual attribute annotation or domain-expert intervention, which may not scale.
- Zero-shot PASTA still lags tracking-by-detection baselines such as ByteTrack on PersonPath22, especially in detection (FN).
Experimental setup
Datasets: MOTSynth MOT17 PersonPath22 DanceTrack
Benchmarks: MOTSynth MOT17 PersonPath22
Metrics: HOTA IDF1 MOTA DetA AssA FP FN IDSW
Baselines: MOTRv2-MS TrackFormer ByteTrack OC-SORT SORT MOTRv2 CenterTrack SiamMOT FairMOT IDFree
Builds on: MOTRv2 LoRA Deformable DETR model merging scale and shift fine-tuning
Closest papers by meaning
- Towards Generalizable Multi-Object Tracking (CVPR 2024 · similarity 0.87)
- Single-Model and Any-Modality for Video Object Tracking (CVPR 2024 · similarity 0.86)
- SUTrack: Towards Simple and Unified Single Object Tracking (AAAI 2025 · similarity 0.85)
- Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline (CVPR 2024 · similarity 0.85)
- UTPTrack: Towards Simple and Unified Token Pruning for Visual Tracking (CVPR 2026 · similarity 0.84)
- OVTR: End-to-End Open-Vocabulary Multiple Object Tracking with Transformer (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.