Research map / Large language models / Language, safety and interpretability
DIFFA: Large Language Diffusion Models Can Listen and Understand
Jiaming Zhou, Hongjie Chen, Shiwan Zhao, Jian Kang, Jie Li, Enzhi Wang, Yujie Guo, Haoqin Sun, Hui Wang, Aobo Kong, Yong Qin, Xuelong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use diffusion LLM (LLaDA) backbone instead of autoregressive LLaMA 3.1 (ablation) | DIFFA (dual adapter, LLaMA3 data) · MMAU | accuracy: 28.40 -> 51.71 | Improved |
| use diffusion LLM (LLaDA) backbone instead of autoregressive LLaMA 3.1 (ablation) | DIFFA (dual adapter, LLaMA3 data) · MMSU | accuracy: 38.40 -> 54.72 | Improved |
| add acoustic adapter (dual vs single adapter) (ablation) | DIFFA · MMAU | accuracy: 47.70 -> 49.71 | Improved |
| add acoustic adapter (dual vs single adapter) (ablation) | DIFFA · MMSU | accuracy: 52.88 -> 56.04 | Improved |
| use Qwen3-generated instruction data instead of LLaMA3 (ablation) | DIFFA · VoiceBench | accuracy: 37.17 -> 48.22 | Improved |
| use Qwen3-generated instruction data instead of LLaMA3 (ablation) | DIFFA · MMAU | accuracy: 51.71 -> 49.71 | Mixed |
| rewrite Qwen3 captions with LLaDA (ablation) | DIFFA · Avg of MMAU/MMSU/VoiceBench | accuracy: 51.32 -> 51.15 | No clear effect |
| use LLaDA-generated instruction data instead of LLaMA3 (ablation) | DIFFA · Avg of MMAU/MMSU/VoiceBench | accuracy: 47.86 -> 50.34 | Improved |
| train DIFFA with 127h synthetic data vs Qwen2-Audio large-scale data | DIFFA · MMSU | accuracy: 53.27 -> 56.04 | Improved |
| DIFFA vs Qwen2-Audio on VoiceBench | DIFFA · VoiceBench overall | score: 55.34 -> 48.22 | Hurt |
Closest papers by meaning
- Listen, Think, and Understand (ICLR 2024 · similarity 0.89)
- MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix (NeurIPS 2025 · similarity 0.89)
- MATS: An Audio Language Model under Text-only Supervision (ICML 2025 · similarity 0.89)
- Measuring Audio's Impact on Correctness: Audio-Contribution-Aware Post-Training of Large Audio Language Models (ICLR 2026 · similarity 0.89)
- DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors (ICLR 2025 · similarity 0.88)
- Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
