Research map / Large language models / Reasoning and chain of thought

RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies RL with verifiable emotion rewards from a SAGE-based simulated user to train Qwen2.5-7B-Instruct for empathetic dialogue, raising its Sentient Benchmark score from 13.3 to 79.2 while largely preserving math and coding ability.

Problem

LLMs lag far behind in emotional intelligence, and RLVR has not been applied to multi-turn empathetic dialogue because stable environments and consistent, verifiable emotion rewards are lacking.

Approach

Use the SAGE self-consistent affective user simulator as a live RL environment, where its deterministic terminal emotion score (normalized to [0,1]) serves as the reward; compare PPO and GRPO and a think-then-say prompt scaffold under this heart-in-the-loop setup.

Key results

• PPO with think scaffold raises Qwen2.5-7B-Instruct Sentient Benchmark score from 13.3 to 79.2, close to Gemini2.5-Pro (82.4).
• GRPO with think reaches 72.0 and PPO without think 61.7 on the Sentient Benchmark; the challenging user simulator lowers scores (thinking PPO 66.4, non-thinking 19.8).
• Best PPO model keeps MATH500 at 76.6 (vs 77.8 base) and improves LiveCodeBench from 26.7 to 28.0; IFEval 68.6 vs 70.4.

Surprising findings

• A more challenging user simulator yields worse training outcomes than the vanilla simulator, with non-thinking PPO dropping from 61.7 to 19.8.
• Non-thinking models show little or no improvement over the base model under the challenging simulator, while thinking models still improve in several capabilities.

Limitations

• Reliance on DeepSeek-V3 as the simulator means rewards depend on an LLM simulator rather than real human users.
• Evaluation on a small test set (100 scenarios) and the Sentient Benchmark itself is LLM-simulated, with a single base model (Qwen2.5-7B).
• Reward hacking mitigation is asserted but the simulator remains an LLM whose behavior may be exploitable.

Experimental setup

Datasets: SAGE scenarios (constructed 500 training / 100 test) MATH500 LiveCodeBench release_v6 IFEval

Benchmarks: Sentient Benchmark (SAGE) Chit Chat (SAGE extension) MATH500 LiveCodeBench IFEval

Metrics: Sentient score success rate failure rate LLM-judged five capability scores (1-5) strategy frequency and strategy contribution pass@1 strict-prompt accuracy

Baselines: Gemini2.5-Pro-0605 GPT-4o-0326 GPT-4.1-0414 Gemini2.5-Flash-Think-0520 OpenAI-o3-0416 Qwen2.5-7B-Instruct

Builds on: SAGE PPO GRPO DeepSeek-R1 verl

Closest papers by meaning

• EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models (CVPR 2026 · similarity 0.90)
• DialogXpert: Driving Intelligent and Emotion-Aware Conversations Through Online Value-Based Reinforcement Learning with LLM Priors (AAAI 2026 · similarity 0.89)
• From Stimuli to Minds: Enhancing Psychological Reasoning in LLMs via Bilateral Reinforcement Learning (AAAI 2026 · similarity 0.88)
• EmotionThinker: Prosody-Aware Reinforcement Learning for Explainable Speech Emotion Reasoning (ICLR 2026 · similarity 0.88)
• Unveiling the Cognitive Compass: Theory-of-Mind–Guided Multimodal Emotion Reasoning (ICLR 2026 · similarity 0.88)
• MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models (ICLR 2026 · similarity 0.87)