Research map / Large language models / Language, safety and interpretability
Using Reinforcement Learning to Train Large Language Models to Explain Human Decisions
Jian-Qiao Zhu, Hanbo Xie, Dilip Arumugam, Robert Wilson, Thomas L. Griffiths
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL with outcome reward instead of SFT | Qwen-2.5-7B-Instruct · choices13k test | MSE: SFT .0144 vs RL .0148 | No clear effect |
| Centaur-style masking instead of plain SFT | Qwen-2.5-7B-Instruct · choices13k test | MSE: .0144 -> .0155 | No clear effect |
| use a weaker backbone (Gemma-2-2B) for RL (ablation) | RL (GRPO) · choices13k test | MSE: .0148 -> 0.0526 | Hurt |
| use a weaker backbone (Gemma-2-2B) for SFT (ablation) | SFT · choices13k test | MSE: .0144 -> 0.0162 | No clear effect |
| replace backbone CoT with RL model's CoT (ablation) | Qwen-2.5-7B-Instruct · choices13k test | MSE: .0694 -> .0212 | Improved |
| replace RL model's CoT with backbone's CoT (ablation) | Qwen-2.5-7B-GRPO-c13k · choices13k test | MSE: .0148 -> .0695 | Hurt |
| feed SFT-generated CoT to the backbone (ablation) | Qwen-2.5-7B-Instruct · choices13k test | MSE: .0694 -> 0.0785 | No clear effect |
| train RL on synthetic expected-value-maximizer choice rates instead of human data (ablation) | RL (GRPO) · CoT mechanisms | dominant CoT mechanism | Improved |
| train RL on random choice rates (ablation) | RL (GRPO) · Random dataset | CoT quality | Hurt |
Closest papers by meaning
- Turning large language models into cognitive models (ICLR 2024 · similarity 0.92)
- Language Models Trained to do Arithmetic Predict Human Risky and Intertemporal Choice (ICLR 2025 · similarity 0.91)
- On the Modeling Capabilities of Large Language Models for Sequential Decision Making (ICLR 2025 · similarity 0.89)
- Eliciting Reasoning in Language Models with Cognitive Tools (NeurIPS 2025 · similarity 0.89)
- Generating Computational Cognitive models using Large Language Models (NeurIPS 2025 · similarity 0.89)
- CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
