Research map / Vision-language and multimodal / Multimodal LLMs

CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces CMI-Pref-Pseudo (110k LLM-labeled pairs), CMI-Pref (4,027 human-annotated pairs), a unified CMI-RewardBench for music reward models, and CMI-RM, a ~30M-parameter two-tower reward model over frozen MuQ-MuLan encoders.

Problem

Music generation models accept mixed text, lyrics, and reference-audio prompts, but existing music evaluators are fragmented: they score musicality alone or alignment for a single modality pair, and lack human-labeled preference data for compositional instructions.

Approach

A two-tower reward model with frozen MuQ-MuLan encoders that fuses optional text, lyric, and reference-audio prompt embeddings with the evaluated audio, outputting separate musicality and alignment scores. It is pretrained on Qwen3-Omni pseudo-preferences with position-consistency filtering and label smoothing, then fine-tuned on human preferences and MusicEval.

Key results

• CMI-RM (fine-tuned on CMI+MusicEval) reaches 78.20% accuracy on CMI-Pref musicality, versus 65.80% for Gemini 3 Pro and 60.40% for Qwen3-Omni.
• On PAM music subset, the CMI-Pref fine-tuned model reaches SRCC 0.6988, exceeding SongEval-RM at 0.6977.
• Best-of-N reranking with the RM improves MusicGen-small MuQ-MuLan alignment from 0.298 (N=1) to 0.339 (N=10).

Surprising findings

• Gemini 3 Pro and Qwen3-Omni reach only around 60-66% on CMI-Pref musicality, below the reward model.
• The reward model agrees with its Qwen3-Omni teacher only 63.7%/68.4% after human fine-tuning, indicating human supervision moves the decision boundary away from the teacher.

Limitations

• Musicality and alignment heads are strongly correlated (SRCC about 0.853), suggesting incomplete disentanglement.
• Performance varies by genre and language, with weaker alignment on low-resource languages.
• Human inter-annotator agreement is moderate (Krippendorff's alpha about 0.5).

Experimental setup

Datasets: CMI-Pref-Pseudo CMI-Pref MusicEval PAM Music Arena

Benchmarks: CMI-RewardBench PAM MusicEval Music Arena CMI-Pref

Metrics: LCC SRCC K-Tau accuracy

Baselines: PAM Audiobox-Aesthetic SongEval CLAP MuQ-MuLan CLAMP3 Omni-Reward Qwen3-Omni Gemini 2.5 Pro Gemini 3 Pro

Builds on: Bradley-Terry MuQ-MuLan

Closest papers by meaning

• MusicRL: Aligning Music Generation to Human Preferences (ICML 2024 · similarity 0.91)
• LeVo: High-Quality Song Generation with Multi-Preference Alignment (NeurIPS 2025 · similarity 0.89)
• LLark: A Multimodal Instruction-Following Language Model for Music (ICML 2024 · similarity 0.88)
• Responding to the Call: Exploring Automatic Music Composition Using a Knowledge-Enhanced Model (AAAI 2024 · similarity 0.88)
• JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generation (AAAI 2025 · similarity 0.87)
• SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation (ICML 2025 · similarity 0.87)