AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition

Evangelos Kazakos, Arsha Nagrani, Andrew Zisserman, Dima Damen

ICCV 2019 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modality image fusion · 331 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add audio to RGB+Flow in TBN (ablation)TBN · EPIC-Kitchens S1top-1 action accuracy: 30.31 -> 34.80Improved
add audio to RGB+Flow in TBN (ablation)TBN · EPIC-Kitchens S2top-1 action accuracy: 16.80 -> 19.06Improved
add audio to RGB+Flow in TBN (ablation)TBN · EPIC-Kitchens S1top-5 action accuracy: 51.81 -> 56.65Improved
add audio to RGB+Flow in TBN (ablation)TBN · EPIC-Kitchens S2top-5 action accuracy: 32.61 -> 36.54Improved
add audio in TBN, segments with irrelevant sounds (ablation)TBN · EPIC-Kitchens S2 irrelevant-audio segmentstop-1 action accuracy: 12.39 -> 15.30Improved
fuse modalities with TBN (All) instead of best single modality (ablation)TBN · EPIC-Kitchens S1top-1 action accuracy: 20.10 -> 34.80Improved
use temporal binding network instead of late fusion TSN with audioTBN single model · EPIC-Kitchens S1top-1 action accuracy: 23.95 -> 34.80Improved
ensemble five TBNs with different TBW widthsTBN · EPIC-Kitchens S1top-1 action accuracy: 34.80 -> 36.66Improved
use context gating instead of concatenation for fusion (ablation)TBN · EPIC-Kitchens S1top-1 action accuracy: 34.80 -> 33.47Hurt
use gating fusion instead of concatenation (ablation)TBN · EPIC-Kitchens S1top-1 action accuracy: 34.80 -> 31.61Hurt
use context gating instead of concatenation (ablation)TBN · EPIC-Kitchens S2top-1 action accuracy: 19.06 -> 19.16No clear effect
vary TBW width during training between T/6, T/3 and T (ablation)TBN · EPIC-Kitchenstop-1 accuracy: 4.21 -> 11.00No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy