Research map / Architectures and efficiency / RNNs and normalization
A Light Weight Model for Active Speaker Detection
Junhua Liao, Haihan Duan, Kanghui Feng, Wanbing Zhao, Yanbing Yang, Liangyin Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use kernel sizes 3 and 5 together instead of kernel size 3 alone (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 93.0 -> 94.1 | Improved |
| Increase kernel size from 5 to 7 (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 93.4 -> 93.4 | No clear effect |
| Replace the lightweight visual encoder with the TalkNet visual encoder (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 94.1 -> 92.8 | Hurt |
| Replace split 2D and 1D visual convolutions with 3D convolution (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 94.1 -> 92.9 | Hurt |
| Replace the lightweight audio encoder with ResNet-18 (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 94.1 -> 93.4 | Hurt |
| Use 2D convolutions instead of split 1D audio convolutions (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 94.1 -> 93.6 | Hurt |
| Add temporal modeling with forward GRU instead of direct FC prediction (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 88.0 -> 92.6 | Improved |
| Use bidirectional GRU instead of forward GRU (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 92.6 -> 94.1 | Improved |
| Replace forward GRU with a transformer detector (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 92.6 -> 91.5 | Hurt |
| Add the proposed loss with visual auxiliary supervision (ablation) | Our Method · AVA-ActiveSpeaker validation | mAP (%): 93.1 -> 94.1 | Improved |
Closest papers by meaning
- How To Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild (ICCV 2021 · similarity 0.92)
- Active Speakers in Context (CVPR 2020 · similarity 0.91)
- MAAS: Multi-Modal Assignation for Active Speaker Detection (ICCV 2021 · similarity 0.91)
- LoCoNet: Long-Short Context Network for Active Speaker Detection (CVPR 2024 · similarity 0.89)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.83)
- $\texttt{AVROBUSTBENCH}$: Benchmarking the Robustness of Audio-Visual Recognition Models at Test-Time (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
