Research map / Large language models / Reasoning and chain of thought

EmotionThinker: Prosody-Aware Reinforcement Learning for Explainable Speech Emotion Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Reformulates speech emotion recognition as RL-driven explainable reasoning, combining a prosody-enhanced SpeechLLM (EmotionThinker-Base), a 35K CoT dataset, and GRPO-PTR, which adds a trust-weighted reasoning reward from a trained reward model.

Problem

Current SpeechLLMs treat speech emotion recognition as classification, giving limited interpretability and weak prosody perception, and standard rule-based RL rewards do not supervise reasoning quality.

Approach

Progressively add a multi-dimensional reasoning reward from a trained reward model, gated by a group-level trustworthiness weight that compares reasoning rewards of correct versus incorrect responses, on top of GRPO with format and accuracy rewards; the base model is first made prosody-aware via prosody-centric SFT.

Key results

• EmotionThinker averages 68.89% emotion accuracy across IEMOCAP, MELD, RAVDESS, and SAVEE, versus 65.41% for the next-best BLSP-Emo.
• EmotionThinker reaches an average reasoning score of 3.98 on the GPT-4o-judged 5-point scale, versus 3.04 for the next-best MERaLiON2.
• Ablation: removing the trained reward model drops SER accuracy to 66.67% and ER score to 3.36; removing the trust weight yields ER 3.74.

Surprising findings

• Standard GRPO (V2) raises accuracy to 62.91% but lowers reasoning score to 3.45, below the SFT variant's 3.78.
• Disabling progressive reward scheduling lowers accuracy to 62.80%, below the GRPO-only variant at 62.91%.

Limitations

• Reasoning quality is judged mainly by GPT-4o, with human evaluation limited to 100 samples from five models.
• Training labels and reasoning traces are synthesized by GPT-4o from automatic prosody annotations, which may carry annotation noise.
• Progressive reward scheduling uses a manually chosen threshold (e.g., 50% accuracy).

Experimental setup

Datasets: EmotionCoT-35K IEMOCAP MELD Expresso MEAD EARS Stress-17K GigaSpeech LibriSpeech RAVDESS SAVEE

Benchmarks: IEMOCAP MELD RAVDESS SAVEE

Metrics: unweighted accuracy reasoning score (1-5) on factual alignment, interpretative q human rating on 100 samples

Baselines: Qwen2.5-Omni-7B Kimi-Audio BLSP-Emo OSUM-EChat MERaLiON2 MiniCPM-O Phi-4-Multimodal Standard GRPO

Builds on: GRPO DeepSeek-R1 Qwen2.5-Omni

Closest papers by meaning

• VowelPrompt: Hearing Speech Emotions from Text via Vowel-level Prosodic Augmentation (ICLR 2026 · similarity 0.90)
• EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
• Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier (AAAI 2026 · similarity 0.89)
• Unveiling the Cognitive Compass: Theory-of-Mind–Guided Multimodal Emotion Reasoning (ICLR 2026 · similarity 0.88)
• RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents (ICLR 2026 · similarity 0.88)
• AVERE: Improving Audiovisual Emotion Reasoning with Preference Optimization (ICLR 2026 · similarity 0.88)