Research map / Vision-language and multimodal / Cross-modal learning
AVTrack: Audio-Visual Tracking in Human-centric Complex Scenes
Yaoting Wang, Yun Zhou, Zipei Zhang, Henghui Ding
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add speech separation with MossFormer2 (ablation) | AVTracker · AVTrack | HOTA: 28.85 -> 29.08 | Improved |
| add speech separation with SepFormer (ablation) | AVTracker · AVTrack | HOTA: 28.85 -> 28.41 | Hurt |
| use larger speech processor (Whisper-large-v3-turbo vs Whisper-small) (ablation) | AVTracker · AVTrack | HOTA: 25.19 -> 28.85 | Improved |
| use larger VLM (Qwen3-VL-8B vs 4B) (ablation) | AVTracker · AVTrack | HOTA: 24.47 -> 28.85 | Improved |
| use VLM global reasoner instead of face detection features for tracklet grouping (ablation) | AVTracker · AVTrack | HOTA: 23.62 -> 24.01 | Improved |
| add local chunk compression (ablation) | AVTracker · AVTrack | HOTA: 16.88 -> 24.01 | Improved |
| add dynamic windowing (ablation) | AVTracker · AVTrack | HOTA: 27.45 -> 28.85 | Improved |
| add audio information in AVIS methods compared with VIS-only methods | AVTrack baselines · AVTrack | HOTA: 11.46 -> 20.84 | Improved |
| use AVTracker modular pipeline instead of strongest AVIS baseline AVTrackFormer | AVTrack · AVTrack | HOTA: 21.47 -> 29.08 | Improved |
Closest papers by meaning
- Audio-Visual Instance Segmentation (CVPR 2025 · similarity 0.89)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.88)
- Improving Audio-Visual Segmentation with Bidirectional Generation (AAAI 2024 · similarity 0.87)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.87)
- Unraveling Instance Associations: A Closer Look for Audio-Visual Segmentation (CVPR 2024 · similarity 0.87)
- Dynamic Derivation and Elimination: Audio Visual Segmentation with Enhanced Audio Semantics (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
