AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Prototype Entropy Alignment: Reinforcing Structured Uncertainty in LLM Reasoning

Zhengyuan Pan, Yanhao Chen, Zhongquan Jian, Wanru Zhao, Haonan Ma, Meihong Wang, Qingqiang Wu

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Prototype Entropy Alignment (PEA), an RL framework that clusters expert trajectories' token-entropy patterns into learnable "entropy signature" prototypes and aligns model reasoning to them.

Problem

Reasoning quality in LLMs is driven by a minority of high-entropy tokens, but existing RL methods do not model the structured uncertainty patterns of effective reasoning as a whole.

Approach

Instead of treating effective reasoning as a single path, model it as a collection of entropy signatures: cluster expert trajectories' uncertainty patterns into prototypes and use them as RL alignment targets.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add dynamic prototype updates (periodic re-clustering) instead of static prototypes (ablation)PEA on Qwen2.5-7B · AIME 2024accuracy: 9.1 -> 11.9Improved
add dynamic prototype updates instead of static prototypes (ablation)PEA on Qwen2.5-7B · NaturalReasoningaccuracy: 21.3 -> 65.8Improved
add dynamic prototype updates (reward variance) (ablation)PEA on Qwen2.5-7B · AIME 2024reward variance: 0.312 -> 0.452Improved
restrict entropy signature to top 20% high-entropy tokens instead of all tokens (ablation)PEA on Qwen2.5-7B · training reward and gainperformance gain and average rewardImproved
use 20% instead of 10% or 30% high-entropy tokens (ablation)PEA on Qwen2.5-7B · training reward and gainperformance gainImproved
add PEA reward to C/I rewardQwen2.5-7B-Instruct · AMC23Pass@1: 45.8 -> 48.7Improved
add PEA reward alone on a standard instruct modelQwen2.5-7B-Instruct · AMC23Pass@1: 42.3 -> 40.3Hurt
add PEA reward alone on a standard instruct modelQwen2.5-1.5B-Instruct · AIMEPass@1: 2.6 -> 4.7Improved
add PEA reward alone on a reasoning-specialized modelDeepSeek-R1-Distill-Qwen-7B · AIMEPass@1: 47.1 -> 55.3Improved
add outcome C/I reward alone on a reasoning-specialized modelDeepSeek-R1-Distill-Qwen-7B · MATH500Pass@1: 93.6 -> 90.6Hurt
add PEA + C/I on a reasoning-specialized modelDeepSeek-R1-Distill-Qwen-7B · AIMEPass@1: 47.1 -> 56.7Improved
add C/I reward alone on a standard instruct modelQwen2.5-1.5B-Instruct · AIMEPass@1: 2.6 -> 16.8Improved

Experimental setup

Datasets: expert reasoning trajectories

Builds on: high-entropy token RL

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy