AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

MaxIM: Maximally Informative Incremental Summarization via Reinforcement Learning

Jihwan Jeong, Guy Tennenholtz, Yinlam Chow, Chih-wei Hsu, Craig Boutilier

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: Reward modeling versus direct optimization · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MaxIM, an RL framework that trains incremental summarization memory for bounded-memory agents by addressing sparse, delayed reward credit assignment.

Problem

Maintaining bounded memory over unbounded interaction histories, where optimizing memory representations and updates for downstream utility via RL is hampered by sparse, delayed rewards and credit assignment bottlenecks.

Approach

Formulates incremental summarization as a sequential decision problem and trains the summarizer with RL, aiming to make each memory update maximally informative for downstream task utility rather than relying on end-of-episode sparse rewards alone.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add MaxIM information-theoretic reward shaping to RLTask-Only RL summarizer · arXiv Abstain-MCQ (K=500)accuracy (%): 54.44 -> 66.39Improved
add recursive consistency reward to sufficiency reward (ablation)MaxIM · arXiv Abstain-MCQ (K=500)accuracy (%): 51.54 -> 66.39Improved
add recursive consistency reward to sufficiency reward (ablation)MaxIM · WildChat groundingcontradiction (%): 6.48 -> 4.27Improved
add task reward to MI shaping (ablation)MI-Only RL · arXiv Abstain-MCQ (K=500)accuracy (%): 50.30 -> 66.39Improved
cosine-anneal the shaping weight mid-training (45-60%) vs none (ablation)MaxIM · arXiv Abstain-MCQ test (K=500)accuracy (%): 57.10 -> 66.39Improved
decay shaping early (13-60%) or late (65-90%) instead of mid (ablation)MaxIM · arXiv Abstain-MCQ test (K=500)accuracy (%): 66.39 -> 61.18 (early), 61.12 (late)Hurt
use constant MI shaping without annealing vs task-only reward (ablation)Task-Only RL · arXiv Abstain-MCQ test (K=500)accuracy (%): 54.44 -> 57.10Improved
train summarizer with sparse-reward RL (Task-Only) instead of SFT/promptingGemini 2.5 Flash summarizer · arXiv Abstain-MCQ (K=500)accuracy (%): 31.30 -> 54.44Improved
select single-step updates by highest immediate PMI (Best-of-N MI SFT)Policy SFT · arXiv Abstain-MCQ (K=500)accuracy (%): 31.30 -> 29.05Hurt
use task-only or MemAgent RL summaries instead of no belief trackingWildChat assistant · WildChat (all T)Satisfaction delta vs No-Belief: -0.30 (Task-Only), -0.28 (MemAgent)Hurt
add MaxIM shaping on WildChat T>=10 dialoguesTask-Only RL · WildChat (T>=10)Satisfaction delta vs No-Belief: -0.37 -> +0.11Improved
increase token budget K from 100 to 800 (ablation)MaxIM · arXiv Abstain-MCQaccuracy (%): 34.26 -> 67.04Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy