Research map / Video / Action recognition
End-to-End Active Speaker Detection
Juan León Alcázar, Moritz Cordes, Chen Zhao, Bernard Ghanem
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add end-to-end training of the encoders (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 91.9 -> 93.5 | Improved |
| add iGNN block (vs joint spatial and temporal message passing in one layer) (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 93.7 -> 94.1 | Improved |
| add residual connections between iGNN blocks (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 93.8 -> 94.1 | Improved |
| increase temporal endpoints from l=1 to l=7 (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 89.0 -> 94.1 | Improved |
| increase endpoints beyond l=7 to l=9 (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 94.1 -> 93.8 | Hurt |
| increase tracklets per endpoint from i=2 to i=4 (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 94.1 -> 93.7 | Hurt |
| use 3 tracklets instead of 2 when only one endpoint is used (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 89.0 -> 89.6 | Improved |
| increase input clip size from 11 to 15 frames (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 93.5 -> 94.1 | Improved |
| stack more iGNN blocks (2 to 4 for ResNet50) (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 93.6 -> 94.1 | Improved |
| change iGNN linear layer width from 128 to 64 or 256 (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 94.1 -> 93.8 | No clear effect |
| fuse spatial and temporal information in two separate streams instead of interleaving (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 94.1 -> 92.8 | Hurt |
| reverse order to temporal-then-spatial message passing (ablation) | EASEE-50 · AVA-ActiveSpeaker val | mAP: 94.1 -> 94.0 | No clear effect |
Closest papers by meaning
- Learning Long-Term Spatial-Temporal Graphs for Active Speaker Detection (ECCV 2022 · similarity 0.91)
- LoCoNet: Long-Short Context Network for Active Speaker Detection (CVPR 2024 · similarity 0.90)
- Active Speakers in Context (CVPR 2020 · similarity 0.90)
- How To Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wild (ICCV 2021 · similarity 0.90)
- Cross-Modal Supervision for Learning Active Speaker Detection in Video (ECCV 2016 · similarity 0.89)
- Separate and Reconstruct: Asymmetric Encoder-Decoder for Speech Separation (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
