Research map / Reinforcement learning / Visual and representation RL
Intrinsic Credit Assignment for Long Horizon Interaction
Ilze Amanda Auzina, Joschka Strüber, Sergio Hernández-Gutiérrez, Shashwat Goel, Ameya Prabhu, Matthias Bethge
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DeltaBelief dense turn-level reward to GRPO (CIA vs StarPO) | Qwen3-4B on 20 Questions · 20 Questions test set | Mean@8 success rate: 24.36% -> 33.72% | Improved |
| add DeltaBelief dense turn-level reward to GRPO (CIA vs StarPO) | Qwen3-1.7B on 20 Questions · 20 Questions test set | Mean@8 success rate: 16.54% -> 24.80% | Improved |
| RL with DeltaBelief on top of SFT baseline | Qwen3-4B · 20 Questions test set | Pass@8: 36.87% -> 63.97% | Improved |
| best-of-8 question sampling selecting max DeltaBelief at inference (ablation) | Qwen3-1.7B, 4B, 8B · 20 Questions | success rate | Improved |
| set DeltaBelief reward weight lambda >= 0.5 (ablation) | CIA · 20 Questions validation | training stability | Hurt |
| use moderate lambda = 0.1 instead of 0.05 (ablation) | CIA · 20 Questions validation | validation performance | Improved |
| replace ReLU shaping with naive, EMA or tanh normalization of DeltaBelief (ablation) | CIA · 20 Questions training | training dynamics | Hurt |
| replace ReLU with PACR (positive plus min-max scaling) (ablation) | CIA · 20 Questions training | training dynamics | No clear effect |
| swap training user simulator for stronger answerers at test time (ablation) | CIA (Qwen3-4B) · 20 Questions test set | Mean@8 success rate: 35.65 -> 31.18 / 37.64 | No clear effect |
| extend interaction budget from 20 to 50 turns | CIA (Qwen3-4B) · 20 Questions test set | success rate: CIA 26% higher absolute than StarPO at 50 turns | Improved |
| CIA vs StarPO on Murder Mystery (OOD) | Qwen3-4B · Murder Mystery | Pass@1: +28.5% over StarPO | Improved |
| CIA vs StarPO on Guess My City (OOD) | Qwen3-4B · Guess My City | Pass@1: +12.6% over StarPO | Improved |
Closest papers by meaning
- Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning (AAAI 2025 · similarity 0.88)
- Retrospective In-Context Learning for Temporal Credit Assignment with Large Language Models (NeurIPS 2025 · similarity 0.88)
- S2T-RLHF: Hierarchical Credit Assignment for Stable Preference-Based RLHF (NeurIPS 2026 · similarity 0.87)
- SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning (ICML 2026 · similarity 0.86)
- Learning to Reason without External Rewards (ICLR 2026 · similarity 0.85)
- Agentic Multi-Turn Reasoning: A Fairness Approach (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
