AI Paper Atlas

Research map / Large language models / Alignment and preferences

Beyond Step Pruning: Information Theory Based Step-level Optimization for Self-Refining Large Language Models

Jinman Zhao, Erxue Min, Hui Wu, Ziheng Li, Zexu Sun, Hengyi Cai, Shuaiqiang Wang, Xu Chen, Gerald Penn

AAAI 2026 · Large language models › Alignment and preferences · approach: Active and token-level preference learning · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
step-level DPO instead of solution-level DPOQwen2.5-7B · Math averageaverage accuracy: 52.5 -> 53.7Improved
prune steps with ISLA score (12% data) vs all steps (ablation)Step-DPO Qwen2.5-7B · Math averageaverage accuracy: 53.7 -> 55.0Improved
prune steps with ISLA score (12% data) vs all steps (ablation)Step-DPO Qwen2.5-3B · Math averageaverage accuracy: 45.5 -> 46.1Improved
prune steps with ISLA score (20% data) vs all steps (ablation)Step-DPO Qwen2.5-1.5B · Math averageaverage accuracy: 38.4 -> 39.2Improved
combine relative accuracy and logp instead of logp only (ablation)ISLA step selection · Math average (Qwen2.5-7B)average accuracy: 52.9 -> 55.0Improved
combine relative accuracy and logp instead of relative accuracy only (ablation)ISLA step selection · Math average (Qwen2.5-7B)average accuracy: 54.2 -> 55.0Improved
select steps by score instead of randomly (ablation)ISLA step selection · Math average (Qwen2.5-7B)average accuracy: 52.0 -> 55.0Improved
random step selection instead of all steps (ablation)Step-DPO Qwen2.5-3B · Math averageaverage accuracy: 45.5 -> 42.6Hurt
increase selected steps from 6k to 8k (ablation)ISLA Qwen2.5-7B · Accuracy vs data sizeaccuracyHurt
ISLA step scoring instead of PRM-based selection (Qwen2.5-Math-7B-PRM800K) (ablation)Qwen2.5-7B · Math averageaverage accuracy: 54.1 -> 55.0Improved
ISLA step scoring instead of PRM-based selection (Llama-PRM800K) (ablation)Qwen2.5-7B · Math averageaverage accuracy: 53.8 -> 55.0Improved
ISLA vs solution-level DPOQwen2.5-7B · GSM8Kaccuracy: 87.3 -> 87.3No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy