Research map / Large language models / Reasoning and chain of thought

From Stimuli to Minds: Enhancing Psychological Reasoning in LLMs via Bilateral Reinforcement Learning

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds StimuliQA, an expert-annotated psychological QA dataset from real-life stimuli, and trains compact Qwen2.5 models with Psy-Interpreter, a trajectory-aware GRPO with a bilateral reward, plus SFT-based continual self-labeling, reporting gains on five OOD psychology/commonsense benchmarks.

Problem

LLMs struggle with psychologically grounded tasks requiring inference of implicit mental states in context-rich, ambiguous narratives, partly due to scarce theory-aligned, expert-labeled training data and uniform reasoning strategies that mismatch task demands.

Approach

Combines expert-annotated psychological stimuli converted to QA pairs with a GRPO variant (T-GRPO) that adds a trajectory cache for reward normalization and a bilateral reward that grants length bonuses only to accurate responses, followed by SFT-based continual self-training on confident self-labeled outputs.

Key results

• Psy-Interpreter-SFT (3B) reaches 66.33 F1 / 58.36% accuracy on ToMbench and 82.82 F1 on SocialIQa, versus 26.6 or lower for most open baselines on the same benchmark.
• On the in-house StimuliQA test set, the bilateral reward raises Qwen2.5-1.5B overall F1 from 34.46 (basic R1) to 39.98.
• StimuliQA-trained Qwen2.5-3B SFT models outperform Llama 3.3 70B- and Mistral 8x7B-generated training sets on the OOD benchmarks in the SFT comparison.

Surprising findings

• Psy-Interpreter-SFT (3B) is reported to match or exceed DeepSeek Reasoner on most tasks, though the 0.5B/1.5B/3B baselines are compared with mismatched metrics and the comparison is not fully controlled.

Limitations

• Training data (StimuliQA) relies on LLM-generated QA pairs and is not stated to be released, limiting verification.
• Comparison with commercial and open baselines uses inconsistent setups and reported metrics, weakening the strength of claims.
• Continual learning uses ground-truth-aligned filtering of predictions, so the self-improvement loop is simulated rather than demonstrated with real user feedback.

Experimental setup

Datasets: StimuliQA ToMbench SimpleToM SocialIQa CosmosQA BIG-Bench Hard LlamaQA MistralQA

Benchmarks: ToMbench SimpleToM SocialIQa CosmosQA BIG-Bench Hard StimuliQA

Metrics: token-level F1 accuracy

Baselines: Simple RL Zoo NuExtract New Merges Serialization DeepSeek-R1 Distilled Nemotron Reasoning Tulu Trained Raspberry Transformer RL GPT-4 nano Claude 3 Haiku DeepSeek Reasoner Llama 3.3 70B Instruct

Builds on: GRPO DeepSeek-R1 Chain-of-Thought prompting

Closest papers by meaning

• Unveiling the Cognitive Compass: Theory-of-Mind–Guided Multimodal Emotion Reasoning (ICLR 2026 · similarity 0.89)
• RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents (ICLR 2026 · similarity 0.88)
• EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models (CVPR 2026 · similarity 0.88)
• MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems (NeurIPS 2025 · similarity 0.87)
• Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action (NeurIPS 2026 · similarity 0.87)
• AutoToM: Scaling Model-based Mental Inference via Automated Agent Modeling (NeurIPS 2025 · similarity 0.87)