Research map / Vision-language and multimodal / Multimodal LLMs
TRIAD: Benchmarking Omni-Modal Ambiguity in Multimodal Large Language Models
Zhaolu Kang, Yidi Wang, Yile Li, Siqi Zeng, Yingjie He, richeng xuan, Zhichao Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TRIAD, a benchmark for testing whether omni-modal LLMs can resolve ambiguity distributed across text, vision, and audio.
Problem
Existing multimodal benchmarks test perception, recognition, or reasoning over determinate inputs, leaving unclear whether omni-modal LLMs can resolve ambiguity spread across text, vision, and audio.
Approach
A benchmark built around cross-modal ambiguity, where inputs are underspecified in one channel and resolved only by evidence from other channels (text, image, audio).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| score by group (all sister queries correct) instead of by query | Gemini 3.1 Pro · TRIAD full input | accuracy (%): 90.4 -> 71.2 | Hurt |
| remove the answer option list (open-ended) (ablation) | Gemini 3.1 Pro · TRIAD | group accuracy (%): 71.2 -> 4.6 | Hurt |
| add chain-of-thought prompting (ablation) | Gemini 3.1 Pro · TRIAD | group accuracy (%): 71.2 -> 72.5 | No clear effect |
| remove text input (ablation) | Gemini 3.1 Pro · TRIAD | group accuracy (%): 71.2 -> 16.4 | Hurt |
| remove image input (ablation) | Gemini 3.1 Pro · TRIAD | group accuracy (%): 71.2 -> 57.2 | Hurt |
| remove audio input (ablation) | Gemini 3.1 Pro · TRIAD | group accuracy (%): 71.2 -> 51.8 | Hurt |
| increase scene group size from 2 to 8 (ablation) | Gemini 3.1 Pro · TRIAD | group accuracy (%): 51.8 -> 8.9 | Hurt |
| increase number of ambiguous modalities from 1 to 3 (ablation) | Gemini 3.1 Pro · TRIAD | query accuracy (%): 95.4 -> 87.8 | Hurt |
| replace audio/image with transcript/description (ablation) | omni-modal models · TRIAD | accuracy | Hurt |
Experimental setup
Benchmarks: TRIAD
Closest papers by meaning
- XModBench: Benchmarking Cross-Modal Capabilities and Consistency in Omni-Language Models (ICLR 2026 · similarity 0.91)
- Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs (NeurIPS 2026 · similarity 0.91)
- OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities (ICLR 2025 · similarity 0.90)
- OmniBench: Towards The Future of Universal Omni-Language Models (NeurIPS 2025 · similarity 0.90)
- Omni-Captioner: Data Pipeline, Models, and Benchmark for Omni Detailed Perception (ICLR 2026 · similarity 0.88)
- OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
