Research map / Vision-language and multimodal / Multimodal LLMs
World-Model Inspired Emotion-aware Token Refinement for Training-Free Multimodal Emotion Recognition
Kejun Liu, Zhe Chen, Yuanyuan Liu, ke Wang, Yibing Zhan, Wei Xiang, Hongyan Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free, world-model-inspired emotion-aware token refinement method to improve multimodal emotion recognition with MLLMs.
Problem
MLLMs for multimodal emotion recognition are unreliable because sparse emotional cues are overwhelmed by redundant context; fine-tuning is costly and chain-of-thought prompting relies on heuristic prompts.
Approach
Refines multimodal tokens in a training-free manner, guided by a world-model-inspired view of emotion, instead of relying on heuristic prompting or fine-tuning (details beyond the abstract are not available).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add NTS (noise-suppressed token selection) (ablation) | frozen InternVL3.5-8B · MER23 (InternVL3.5-8B) | F1 (%): 63.26 -> 67.84 | Improved |
| add STR (state-strengthened token reweighting) (ablation) | frozen InternVL3.5-8B · MER23 (InternVL3.5-8B) | F1 (%): 63.26 -> 66.22 | Improved |
| add STR on top of NTS (ablation) | WETR · MER23 (InternVL3.5-8B) | F1 (%): 67.84 -> 68.48 | Improved |
| use contrastive query scoring instead of random token selection (ablation) | NTS · MER23 (InternVL3.5-8B) | UAR (%): 57.07 -> 63.24 | Improved |
| use contrastive scoring instead of Qpos alone (ablation) | NTS · MER23 (InternVL3.5-8B) | UAR (%): 61.48 -> 63.24 | Improved |
| use contrastive Qpos-Qneg instead of Qneg only (ablation) | NTS · MER23 (InternVL3.5-8B) | F1 mean (%): 65.99 -> 68.17 | Improved |
| add anchor protection to STR (ablation) | WETR · MER23 (InternVL3.5-8B) | F1 (%): 68.07 -> 68.48 | Improved |
| add STR to NTS under 30% post-NTS noise (ablation) | WETR · MER23 (InternVL3.5-8B) | F1 drop: -2.70 -> -1.76 | Improved |
| apply WETR at highest noise level (eps=7) | InternVL3.5-8B · MER23 (InternVL3.5-8B) | F1: +9.18% | Improved |
| apply WETR to InternVL3.5-8B | InternVL3.5-8B · MAFW | WAR (%): 55.08 -> 59.64 | Improved |
| apply WETR to InternVL3.5-8B | InternVL3.5-8B · MER23 | WAR (%): 60.34 -> 68.13 | Improved |
| apply WETR to Qwen3-VL-8B | Qwen3-VL-8B · MAFW | WAR (%): 52.78 -> 56.79 | Improved |
Experimental setup
Baselines: chain-of-thought prompting
Closest papers by meaning
- Catch Your Emotion: Sharpening Emotion Perception in Multimodal Large Language Models (ICML 2025 · similarity 0.90)
- Emotion-Coherent Reasoning for Multimodal LLMs via Emotional Rationale Verifier (AAAI 2026 · similarity 0.90)
- Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning (NeurIPS 2024 · similarity 0.90)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.89)
- MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models (ICLR 2026 · similarity 0.89)
- OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
