AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

VocalGrad: Evaluating Acoustic Perception in Audio Language Models

Koki Ryu, Hitomi Yanaka

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Speech and audio language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VocalGrad, a benchmark for isolating acoustic perception in large audio language models by separating it from higher-level reasoning, motivated by LALM failures on emotion recognition.

Problem

Large audio language models struggle to reliably use paralinguistic and non-linguistic audio information (e.g., in emotion recognition), and existing benchmarks entangle acoustic perception with higher-level reasoning, obscuring the source of failures.

Approach

A benchmark designed to evaluate acoustic perception in audio language models on its own, disentangled from reasoning demands. Only the abstract is available, so the specific construction details are not stated.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply transformations to speech clips and ask LALMs for change directionfive LALMs · VocalGradaccuracy (%): 94.4 (humans) vs 51.8 (best model, Kimi-Audio)Hurt
remove linguistic content (use pure sine-wave clips) (ablation)LALMs on VocalGrad · VocalGrad sine-wave subsets (pitch, volume, noise, vibration)accuracy (%): MiMo-Audio noise 53.2 -> 64.2; none above 70%Mixed
add LoRA on the LM head only (ablation)Kimi-Audio · VocalGrad volume/noise/roughnessaccuracy (%): 88-99% on volume, noise, roughnessImproved
add LoRA on the LM head only (ablation)Kimi-Audio · VocalGrad distortion/vibration/pitch/echo/clarityaccuracy (%): 52.2 distortion, 53.3 vibration, 57.0 pitchNo clear effect
apply LoRA to all linear layers instead of the LM head only (ablation)Kimi-Audio · VocalGrad (vibration)accuracy (%): 53.3 -> 99.2Improved
apply LoRA to all linear layers instead of the LM head only (ablation)Kimi-Audio · VocalGrad (speed)accuracy (%): 73.2 -> 99.0Improved
fine-tune on one category, test on other categoriesKimi-Audio LoRA · VocalGrad cross-categoryaccuracy (%)Mixed
use rule-based parsing instead of exact match (ablation)Gemini 3 Flash evaluation · VocalGrad parsingparsed output rate (%): 83.9 -> 99.1Improved
change temporal trajectory type of the transformation (ablation)LALMs · VocalGrad by trajectoryaccuracy (%)No clear effect
increase transformation intensity tier (ablation)LALMs · VocalGrad background noiseaccuracy (%)Improved
decrease transformation intensity tier (ablation)Human annotators · VocalGrad voice pitchaccuracy (%)Hurt

Limitations

Experimental setup

Benchmarks: VocalGrad

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy