Research map / Vision-language and multimodal / Multimodal LLMs

AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AVI-Bench, a cognitively inspired benchmark (perception, understanding, reasoning, plus a primitive-sensation extension AVI-Bench-PriSe) for evaluating 28 open- and closed-source Omni-MLLMs, and proposes a four-level AVI taxonomy scoring task, modality, stage, and domain adaptivity.

Problem

Omni-MLLMs' audio-visual intelligence lacks systematic, cognitively aligned evaluation; existing audio-visual benchmarks are fragmented across tasks and do not diagnose balance across modalities, cognitive stages, or unfamiliar domains.

Approach

Organizes audio-visual tasks into a human-cognition-inspired staged benchmark with a primitive-sensation extension on low-semantic unfamiliar stimuli, and defines a four-level taxonomy (task, modality balance, stage bottleneck, domain adaptation) built from headroom-normalized scores and penalty terms.

Key results

• Gemini-2.5-Pro is the top model with an average of 57.21 on AVI-Bench, while Gemini-2.5-Pro reaches only 39.13 on AVL and 35.08 on AVLG grounding tasks.
• No open-source model exceeds 25.0% on either AVL or AVLG.
• Human participants average 92.6 on AVI-Bench versus 57.2 for Gemini-2.5-Pro in a six-person pilot study; Gemini-2.5-Pro gets 16.50 on AVSQA versus 90.55 for humans.

Surprising findings

• Smaller models such as Human-Omni-0.5B outperform larger counterparts on ASQA without double-confirmation, a counterintuitive scale inversion.
• Many models perform similarly or better with unimodal inputs than multimodal on AVLG and AVQA, indicating weak cross-modal synergy.

Limitations

• Primitive-sensation tasks use MCQ format, which may allow hallucination or guessing despite double-confirmation.
• Human study uses only six participants on a subset of tasks.

Experimental setup

Datasets: AVQA Music-AVQA AV-Odyssey OmniBench AVHBench

Benchmarks: AVI-Bench AVI-Bench-PriSe

Metrics: accuracy mIoU FENSE METEOR ROUGE_L CIDEr SBERT-similarity

Baselines: GPT-4o Gemini-2.5-Pro Gemini-2.5-Flash Gemini-2.0-Flash Qwen2.5-Omni Ola Baichuan-Omni-1.5 Human-Omni R1-Omni Phi-4-Multimodal

Builds on: AV-Odyssey OmniBench AVHBench SAVEBench AVTrustBench

Closest papers by meaning

• HAVE-Bench: Hierarchical Audio-Visual Evaluation from Perception to Interaction (CVPR 2026 · similarity 0.92)
• Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception (ICLR 2026 · similarity 0.91)
• JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation (ICLR 2026 · similarity 0.91)
• From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs (NeurIPS 2026 · similarity 0.90)
• AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models (ICLR 2025 · similarity 0.89)
• OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs (ICLR 2026 · similarity 0.89)