Research map / Vision-language and multimodal / Cross-modal learning
Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parser
Yung-Hsuan Lai, Yen-Chun Chen, Yu-Chiang Frank Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HAN as dense labeler instead of CLIP and CLAP (ablation) | Valor · AVVP LLP val | segment Type@AV: 63.4 -> 56.8 | Hurt |
| modality-agnostic segment labels instead of modality-aware (ablation) | Valor · AVVP LLP val | segment Type@AV: 63.4 -> 44.9 | Hurt |
| replace video-level labels by refined labels in global loss (ablation) | HAN baseline · AVVP LLP val | segment Type@AV: 54.0 -> 60.8 | Improved |
| add CLIP/CLAP segment-level dense labels (ablation) | Valor · AVVP LLP val | segment Type@AV: 55.6 -> 63.4 | Improved |
| use CLIP and CLAP features instead of ResNet-152 and VGGish (ablation) | Valor · AVVP LLP val | segment Type@AV: 64.9 -> 68.6 | Improved |
| thinner yet deeper HAN (4 layers, 256-dim) (ablation) | Valor · AVVP LLP val | segment Type@AV: 63.4 -> 64.9 | Improved |
| distill dense labels with KD loss instead of Valor labels (ablation) | HAN baseline · AVVP LLP val | segment Type@AV: 55.6 -> 54.3 | Hurt |
| remove video-level labels as filters (audio) (ablation) | Valor · AVVP LLP val | audio F-score: 63.4 -> 47.9 | Hurt |
| remove video-level labels as filters (visual) (ablation) | Valor · AVVP LLP val | visual F-score: 65.9 -> 53.8 | Hurt |
| class-dependent instead of unified thresholds (ablation) | Valor · AVVP LLP val | segment Type@AV: 63.1 -> 63.4 | No clear effect |
| apply pseudo label denoising (PLD) to Valor labels (ablation) | Valor · LLP val visual segment fidelity | F-score: 72.34 -> 73.84 | Improved |
| apply PLD before training (ablation) | Valor · AVVP LLP test | segment Type@AV: 62.0 -> 62.2 | No clear effect |
Closest papers by meaning
- Exploring Heterogeneous Clues for Weakly-Supervised Audio-Visual Video Parsing (CVPR 2021 · similarity 0.91)
- Watch, Listen and Tell: Multi-Modal Weakly Supervised Dense Event Captioning (ICCV 2019 · similarity 0.91)
- Adapting to the Unknown: Training-Free Audio-Visual Event Perception with Dynamic Thresholds (CVPR 2025 · similarity 0.89)
- Dense Audio-Visual Event Localization Under Cross-Modal Consistency and Multi-Temporal Granularity Collaboration (AAAI 2025 · similarity 0.89)
- Audio-Visual Semantic Graph Network for Audio-Visual Event Localization (CVPR 2025 · similarity 0.88)
- LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videos (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
