Research map / Vision-language and multimodal / Cross-modal learning
C3ASD: Multi-Level Consistency-Driven Representation Learning for Robust Active Speaker Detection
Jin Hong, Jisoo Park, Junseok Kwon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inter-modality consistency loss (ablation) | C3ASD (Light-ASD) · AVA val | mAP: 93.61 -> 93.70 | Improved |
| add intra-modality track-aware contrastive loss (ablation) | C3ASD (Light-ASD) · AVA val | mAP: 93.61 -> 93.62 | No clear effect |
| add prediction-level consistency loss (ablation) | C3ASD (Light-ASD) · AVA val | mAP: 93.61 -> 93.68 | Improved |
| combine all three consistency losses (ablation) | C3ASD (Light-ASD) · AVA val | mAP: 93.61 -> 93.80 | Improved |
| add consistency losses under object occlusion | C3ASD (Light-ASD) · AVA val corrupted | mAP: 76.86 -> 78.90 | Improved |
| add consistency losses under pixelation | C3ASD (Light-ASD) · AVA val corrupted | mAP: 93.15 -> 93.47 | No clear effect |
| add consistency losses under MUSAN/DEMAND audio noise | C3ASD (Light-ASD) · AVA val corrupted | mAP: 90.2 -> 90.7 (DEMAND) | Improved |
| add consistency losses under joint occlusion and MUSAN noise | C3ASD (Light-ASD) · AVA val corrupted | mAP: +1.23% average mAP | Improved |
| add inter-modality loss: audio-visual cosine similarity (ablation) | C3ASD (Light-ASD) · AVA val | cosine similarity: 0.555 -> 0.834 | Improved |
| add prediction loss under SNR -10 dB noise (ablation) | C3ASD (Light-ASD) · AVA val | AV/V prediction disagreement: 0.324 -> 0.162 | Improved |
| add consistency losses on cross-dataset transfer | C3ASD (Light-ASD) · WASD | mAP: 85.3 -> 86.1 | Improved |
| add intra-modality loss without cross-modal anchoring (appendix PCA) (ablation) | C3ASD · AVA val embeddings | mean paired distance: 0.276 -> 0.886 | Hurt |
Closest papers by meaning
- ES3: Evolving Self-Supervised Learning of Robust Audio-Visual Speech Representations (CVPR 2024 · similarity 0.87)
- LoCoNet: Long-Short Context Network for Active Speaker Detection (CVPR 2024 · similarity 0.86)
- Seeing Speech and Sound: Distinguishing and Locating Audio Sources in Visual Scenes (CVPR 2025 · similarity 0.85)
- Robust Audio-Visual Segmentation via Audio-Guided Visual Convergent Alignment (CVPR 2025 · similarity 0.85)
- Asymmetric Hierarchical Anchoring for Robust Audio–Visual Cross-Modal Generalization (NeurIPS 2026 · similarity 0.85)
- Who Says What: Symbolic Trimodal Binding Mechanisms in Audio-Visual LLMs (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
