Research map / Reinforcement learning / RL from feedback

AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: RL for LLM reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies GRPO-based reinforcement learning with emotion-wheel metric rewards, length penalties, and reasoning-alignment/perception rewards on top of an AffectGPT cold-start model to improve open-vocabulary and basic emotion recognition.

Problem

Open-vocabulary multimodal emotion recognition is trained with token-level losses that misalign with emotion-wheel (EW) set-level metrics, which are non-differentiable and cannot be optimized by backpropagation.

Approach

Optimize EW-based metrics directly as RL rewards with GRPO after a cold-start phase, add length penalties to counter reward hacking, and study multiple reward designs and whether explicit thinking helps.

Key results

• On OV-MERD+ the RL model reaches 68.39 EW score versus 62.84 for AffectGPT trained on the same two-stage data, indicating gains come from RL.
• On MER-UniBench audio+video+text input, AffectGPT-RL achieves a mean of 79.48 versus 74.77 for AffectGPT, with gains on MER2024 (93.75 vs 78.80) and OV-MERD+ (68.05 vs 62.52).
• Sentiment analysis slightly declines under RL (e.g., MOSI 79.39 vs 81.30 for AV+T input).

Surprising findings

• Removing the explicit thinking process improves performance, so the reasoning output hurts OV-MER in this setup.
• Adding more rewards (all five) degrades performance relative to the best reward combination.

Limitations

• Explicit thinking traces from coarse-grained data appear to introduce noise and reduce performance; the reasoning design is not validated as helpful.
• Sentiment analysis performance degrades slightly under RL.
• Perception reward based on an MLLM judge underperforms, likely due to reward noise; RL data scaling not saturated.

Experimental setup

Datasets: MER-Caption+ MER2025-OV OV-MERD+ MER-UniBench MOSI MOSEI SIMS SIMS v2 MER2023 MER2024 MELD IEMOCAP

Benchmarks: OV-MERD+ MER-UniBench

Metrics: EW-based set-level F1 (averaged over five emotion wheels) accuracy (sentiment and basic emotion tasks) mean across tasks

Baselines: AffectGPT R1-Omni Emotion-LLaMA SALMONN Qwen-Audio mPLUG-Owl Chat-UniVi VideoChat

Builds on: GRPO AffectGPT EmoPrefer Bradley-Terry model

Closest papers by meaning

• AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.91)
• OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition (ICML 2025 · similarity 0.91)
• Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning (NeurIPS 2024 · similarity 0.87)
• Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis (AAAI 2026 · similarity 0.87)
• MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models (ICLR 2026 · similarity 0.86)
• EmotionThinker: Prosody-Aware Reinforcement Learning for Explainable Speech Emotion Reasoning (ICLR 2026 · similarity 0.86)