Research map / Vision-language and multimodal / Cross-modal learning
EgoAVU: Egocentric Audio-Visual Understanding
Ashish Seth, Xinhao Mei, Changsheng Zhao, Varun Nagaraja, Ernie Chang, Gregory P. Meyer, Gael Le Lan, Yunyang Xiong, Vikas Chandra, Yangyang Shi, Dinesh Manocha, Zhipeng Cai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune Qwen2.5-Omni-7B on EgoAVU-Instruct | Qwen2.5-Omni-7B · EgoAVU-Bench SSA | LLM-judge score: 1.50 -> 3.20 | Improved |
| fine-tune Qwen2.5-Omni-7B on EgoAVU-Instruct | Qwen2.5-Omni-7B · EgoAVU-Bench AVH | accuracy: 42.69 -> 61.69 | Improved |
| fine-tune on EgoAVU-Instruct (LoRA) | Qwen2.5-Omni-7B · EgoTempo | accuracy: 16.25 -> 20.83 | Improved |
| fine-tune on EgoAVU-Instruct (LoRA) | Qwen2.5-Omni-7B · EgoIllusion | accuracy: 56.32 -> 60.36 | Improved |
| fine-tune on EgoAVU-Instruct (LoRA or full) | Qwen2.5-Omni-7B · EgoSchema | accuracy: 67.43 -> 67.34 (LoRA), 66.21 (Full) | No clear effect |
| full fine-tuning instead of LoRA (ablation) | EgoAVU-Instruct tuning · EgoAVU-Bench TR | accuracy: 64.31 -> 67.84 | Improved |
| full fine-tuning instead of LoRA (ablation) | EgoAVU-Instruct tuning · EgoAVU-Bench AVH | accuracy: 61.69 -> 60.12 | Hurt |
| LoRA instead of full fine-tuning on transfer (ablation) | EgoAVU-Instruct tuning · EgoTempo | accuracy: 20.21 -> 20.83 | Improved |
| fine-tune on EgoAVU-Instruct | Qwen2.5-Omni-7B · EgoAVU-Bench TR (sound subset) | accuracy: 36.17 -> 53.17 | Improved |
Closest papers by meaning
- EgoSound: Benchmarking Sound Understanding in Egocentric Videos (CVPR 2026 · similarity 0.91)
- SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos (CVPR 2024 · similarity 0.91)
- The Audio-Visual Conversational Graph: From an Egocentric-Exocentric Perspective (CVPR 2024 · similarity 0.90)
- EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs (NeurIPS 2025 · similarity 0.89)
- MMEgo: Towards Building Egocentric Multimodal LLMs for Video QA (ICLR 2025 · similarity 0.89)
- Sound Bridge: Associating Egocentric and Exocentric Videos via Audio Cues (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
