AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Mind the Gap: Dataset and Fine-grained Evaluation for Inline Audio Descriptions

Subhashini Venugopalan, Yingwen Tan, Taylor Roper, Jimmy Tobin, Anton Kast, Alicia Martin, Sam Sepah, Amy Pavel

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces a dataset and fine-grained evaluation framework for inline audio descriptions (ADs that must fit within silences) in diverse user-generated video content, targeting MLLM performance.

Problem

MLLM-generated audio descriptions for blind and low-vision users must fit inside existing silences in videos (inline ADs), but their performance on this setting, especially for diverse user-generated content, is under-explored.

Approach

A dataset of inline audio descriptions for user-generated content paired with a fine-grained evaluation protocol for assessing MLLM-generated inline ADs; specific method details are not available from the abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add expert AD guidelines to the prompt (ablation)Gemini-2.5 Flash · MTG-AD validationAudio Overlaps (s): 15.07 -> 59.19Hurt
add expert AD guidelines to the prompt (ablation)Gemini-2.5 Flash · MTG-AD validationOn-Screen Text: 0.47 -> 0.69Improved
add timestamped ASR transcript and speaking-rate target (ablation)Gemini-2.5 Flash · MTG-AD validationAudio Overlaps (s): 59.19 -> 21.34Improved
add explicit per-gap duration and word budget steps (ablation)Gemini-2.5 Flash · MTG-AD validationAudio Overlaps (s): 21.34 -> 12.77Improved
add explicit per-gap word budget on top of Guideline + ASR (ablation)Gemini-2.5 Flash · MTG-AD validationAvg. Silence (s): 6.71 -> 8.97Hurt
use next-generation omni-modal model (Gemini-3.5 Flash)Gemini-2.5 Flash · MTG-AD testAudio Overlaps (s): 8.92 -> 0.26Improved
use Qwen-3.5-Omni-Flash instead of Gemini-2.5 FlashMLLM AD generator · MTG-AD testAudio Overlaps (s): 8.92 -> 52.14Hurt
use reference-free Quality-RF instead of reference-based VideoA11y-Ref AccurateLLM judge · BLV perceived accuracySpearman rho: -0.264 -> +0.244Improved
use Quality-RF instead of VideoA11y-Ref for DescriptiveLLM judge · BLV descriptive ratingSpearman rho: +0.340 -> +0.583Improved
replace human AD with AutoAD (Gemini-2.5 Flash, Gd. + Timing ASR)audio description source · BLV studypreference: 53.6% preferred humanMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy