Research map / Vision-language and multimodal / CLIP and zero-shot
CompA: Addressing the Gap in Compositional Reasoning in Audio-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CompA-order and CompA-attribute expert-annotated benchmarks for compositional reasoning in audio-language models, and CompA-CLAP, a CLAP fine-tune using composition-aware hard negatives and modular contrastive learning.
Problem
Audio-language models like CLAP are largely unable to reason compositionally (event order and attribute binding), and existing audio retrieval benchmarks and training data do not measure or teach this.
Approach
Winoground-style twin-caption benchmarks for audio (order and attribute binding) plus a two-stage contrastive fine-tuning of CLAP: LLM-generated hard negatives that are ignored by other batch items, followed by a modular contrastive loss on template-based synthetic compositional audio with multiple positives and negatives per audio.
Key results
- CompA-CLAP improves over CLAP on CompA-order group score (33.85 vs 11.50) and CompA-attribute group score (15.13 vs 14.75 for CLAP ours), with CompA-order text score 40.70 vs 33.75.
- Existing ALMs (MMT, ML-ACT, CLAP, CLAP-LAION) perform near random on CompA-order and CompA-attribute, with group scores of 3.90 to 5.50 on CompA-order versus random 16.67.
- CompA-CLAP retains retrieval and zero-shot classification performance close to vanilla CLAP (e.g., ESC-50 89.1 vs 90.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune CLAP with compositionally-aware hard negatives followed by modular contrastive learning | CLAP · CompA-order | Group score: 11.50_{\pm0.15} -> 33.85_{\pm0.15} | Helped |
| fine-tune CLAP with compositionally-aware hard negatives followed by modular contrastive learning | CLAP · CompA-attribute | Group score: 14.75_{\pm0.13} -> 15.13_{\pm0.09} | Helped |
| add compositionally-aware hard-negative training (ablation) | CLAP with modular contrastive training · CompA-order | Group score: 21.25_{\pm0.10} -> 33.85_{\pm0.15} | Helped |
| add compositionally-aware hard-negative training (ablation) | CLAP with modular contrastive training · CompA-attribute | Group score: 13.04_{\pm0.21} -> 15.13_{\pm0.09} | Helped |
| add modular contrastive training (ablation) | CLAP fine-tuned with hard negatives · CompA-order | Group score: 20.20_{\pm0.05} -> 33.85_{\pm0.15} | Helped |
| add modular contrastive training (ablation) | CLAP fine-tuned with hard negatives · CompA-attribute | Group score: 11.35_{\pm0.19} -> 15.13_{\pm0.09} | Helped |
Surprising findings
- All models, including CompA-CLAP, perform worse than the random baseline on CompA-attribute group score.
- CLAP achieves similar retrieval performance on Clotho and AudioCaps even when word order in captions is shuffled, indicating these benchmarks do not measure compositionality.
Limitations
- CompA-attribute uses synthetic WavJourney audio for some instances, and benchmarks are small (400 and 200 instances).
- Only two-event order and attribute binding are covered; audios with more than two events are left to future work.
- CompA-CLAP underperforms on long-tailed acoustic events and on prepositions unseen during training.
Experimental setup
Datasets: LAION-audio-630K AudioSet AudioSet Strong Clotho AudioCaps ESC-50 US8K VGGSound FSD50K WavJourney-generated audio
Benchmarks: CompA-order CompA-attribute AudioCaps Clotho ESC-50 US8K VGGSound FSD50K
Metrics: text score audio score group score R@1 R@5 R@10 top-1 zero-shot accuracy
Baselines: MMT ML-ACT CLAP CLAP-LAION Wav2CLIP AudioCLIP
Builds on: CLAP Winoground CLIP hard negative fine-tuning
Closest papers by meaning
- Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities (ICML 2025 · similarity 0.89)
- MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.89)
- MECAT: A Multi-Experts Constructed Benchmark for Fine-Grained Audio Understanding Tasks (ICML 2026 · similarity 0.88)
- Mellow: a small audio language model for reasoning (NeurIPS 2025 · similarity 0.88)
- Unified Synthesis of Compositional Speech and Sound from Free-Form Text Prompts (NeurIPS 2026 · similarity 0.87)
- AudSemThinker: Enhancing Audio-Language Models Through Reasoning over Semantics of Sound (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.