Research map / Vision-language and multimodal / Cross-modal learning
EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition
Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio to RGB+Flow in TBN (ablation) | TBN · EPIC-Kitchens S1 | top-1 action accuracy: 30.31 -> 34.80 | Improved |
| add audio to RGB+Flow in TBN (ablation) | TBN · EPIC-Kitchens S2 | top-1 action accuracy: 16.80 -> 19.06 | Improved |
| add audio to RGB+Flow in TBN (ablation) | TBN · EPIC-Kitchens S1 | top-5 action accuracy: 51.81 -> 56.65 | Improved |
| add audio to RGB+Flow in TBN (ablation) | TBN · EPIC-Kitchens S2 | top-5 action accuracy: 32.61 -> 36.54 | Improved |
| add audio in TBN, segments with irrelevant sounds (ablation) | TBN · EPIC-Kitchens S2 irrelevant-audio segments | top-1 action accuracy: 12.39 -> 15.30 | Improved |
| fuse modalities with TBN (All) instead of best single modality (ablation) | TBN · EPIC-Kitchens S1 | top-1 action accuracy: 20.10 -> 34.80 | Improved |
| use temporal binding network instead of late fusion TSN with audio | TBN single model · EPIC-Kitchens S1 | top-1 action accuracy: 23.95 -> 34.80 | Improved |
| ensemble five TBNs with different TBW widths | TBN · EPIC-Kitchens S1 | top-1 action accuracy: 34.80 -> 36.66 | Improved |
| use context gating instead of concatenation for fusion (ablation) | TBN · EPIC-Kitchens S1 | top-1 action accuracy: 34.80 -> 33.47 | Hurt |
| use gating fusion instead of concatenation (ablation) | TBN · EPIC-Kitchens S1 | top-1 action accuracy: 34.80 -> 31.61 | Hurt |
| use context gating instead of concatenation (ablation) | TBN · EPIC-Kitchens S2 | top-1 action accuracy: 19.06 -> 19.16 | No clear effect |
| vary TBW width during training between T/6, T/3 and T (ablation) | TBN · EPIC-Kitchens | top-1 accuracy: 4.21 -> 11.00 | No clear effect |
Closest papers by meaning
- SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos (CVPR 2024 · similarity 0.90)
- Sound Bridge: Associating Egocentric and Exocentric Videos via Audio Cues (CVPR 2025 · similarity 0.87)
- Test-Time Adaptation for Combating Missing Modalities in Egocentric Videos (ICLR 2025 · similarity 0.87)
- Unveiling the Power of Audio-Visual Early Fusion Transformers with Dense Interactions through Masked Modeling (CVPR 2024 · similarity 0.86)
- Learning Spatial Features from Audio-Visual Correspondence in Egocentric Videos (CVPR 2024 · similarity 0.86)
- TIM: A Time Interval Machine for Audio-Visual Action Recognition (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
