Research map / Large language models / Reasoning and chain of thought

Revealing Behavioral Plasticity in Large Language Models: A Token-Conditional Perspective

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows LLMs can be steered at inference by token-prefix conditioning and proposes ToCoRL, an RL method that uses token-conditional rollouts and a customized KL term to internalize direct-answer behavior for factual QA while preserving math reasoning.

Problem

Large reasoning models excel at math through step-by-step reasoning but underperform on factual QA, and inference-time behavior steering via prefixes is transient and requires pre-generated examples.

Approach

Token-conditional generation (forcing a short direct-answer prefix from an instruct model) exposes latent behavioral plasticity; ToCoRL folds this into RL by mixing token-conditioned rollouts with normal rollouts and using a customized KL term (equivalent to a correctness-weighted policy gradient) to guide exploration toward the desired behavior.

Key results

• ToCoRL raises Qwen3-30B-A3B-2507-Thinking SimpleQA accuracy from 18.9 to 28.3, versus 23.6 for GRPO and 23.9 for Adaptive-Thinking.
• On AIME'25, ToCoRL improves accuracy from 80.5 to 81.5, while Adaptive-Thinking drops to 78.9 and Instruct-Tuning GRPO to 80.4.
• Token-conditional generation alone raises SimpleQA accuracy from 18.9% to 20.7% and reduces average response length from 1255 to 891 tokens.

Surprising findings

• Forcing just a few direct-answer tokens at inference improves the Thinking model's factual QA accuracy without any training or new knowledge.
• Token prefixes from a weaker instruct model (Qwen2.5-14B) yield nearly the same ToCoRL gains as from the 235B model.

Limitations

• Evaluated only on one test case (factual QA vs. math reasoning) and on Qwen3-based models; broader tasks not tested.
• Reward relies on an LLM judge (gpt-4o), and a less capable judge may be reward-hacked by overly long responses.
• Compute and hardware details not reported.

Experimental setup

Datasets: SimpleQA AA-Omniscience AIME'24 AIME'25 10K factual query-answer pairs 10K math query-answer pairs

Benchmarks: SimpleQA AA-Omniscience AIME'24 AIME'25

Metrics: accuracy (avg@32 for math) accuracy (avg@8 for SimpleQA token-conditional study) average and median response length

Baselines: GRPO Adaptive-Thinking Instruct-Tuning GRPO

Builds on: GRPO REINFORCE SimpleQA

Closest papers by meaning

• T1: Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling (ICML 2025 · similarity 0.90)
• Attention Illuminates LLM Reasoning: The Uncovered Preplan-and-Anchor Rhythm Enables Fine-Grained Policy Optimization (ICML 2026 · similarity 0.89)
• Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.89)
• $\textbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving (ICLR 2026 · similarity 0.89)
• Markovian Transformers for Informative Language Modeling (ICLR 2026 · similarity 0.89)
• State of Thought Enables Endogenous Reasoning (NeurIPS 2026 · similarity 0.89)