Research map / Large language models / Reasoning and chain of thought

Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BREP, a ReFT variant for math reasoning that truncates training to early tokens, intervenes only on the first k generated tokens, and uses a PID controller to constrain intervention-vector magnitude.

Problem

Representation finetuning (ReFT) methods underperform weight-based PEFT (LoRA) on mathematical reasoning, attributed to misleading early reasoning prefixes and intervention vectors that disturb numerical encoding and accumulate errors during chain-of-thought generation.

Approach

Combines prefix-only training (truncated responses), early-stage-only intervention at inference, and a PID-controlled bias-magnitude constraint on the learned ReFT bias vectors, applied on top of learnable scaling-and-bias representation interventions.

Key results

• On Llama3.1-8B-Instruct, BREP reaches 87.2 average on Math10K benchmarks vs 85.2 for LoRA and 76.7 for LoReFT; on MATH500 it reaches 63.5 vs 43.7 for LoRA.
• On Qwen3-14B, BREP averages 87.3 on MATH500/AMC23 vs 77.2 for LoRA and 75.3 for LoReFT.
• Standard ReFT bias norm exceeds 1.07, associated with a 6.2% GSM8K drop; BREP's learned bias norm converges near 0.608 on Llama.

Surprising findings

• Vanilla ReFT (RED, LoReFT) falls below the base model on most math benchmarks, even though it outperforms PEFT on other tasks.
• On Qwen3-8B, RED drops MATH500-related performance sharply (54.2 on MATH500 vs base 82.0).

Limitations

• Bias-magnitude targets must be set manually per model family (1.0 for Llama, 1.5 for Qwen), and optimal magnitudes vary across models.
• Cross-architecture bias vectors are near-orthogonal, so learned interventions are not transferable across model families.
• Error analysis and numerical probe results are based on a single model (Llama3-8B-Instruct) and small sample sizes (100 GSM8K cases, 21 numerical errors).

Experimental setup

Datasets: Math10K PRM800K Commonsense170K Synthetic number probe dataset (9,000 addition problems)

Benchmarks: GSM8K SVAMP MathQA MATH500 AMC23 BoolQ PIQA GPQA

Metrics: accuracy

Baselines: Base LoRA RED LoReFT

Builds on: LoReFT RED LoRA Chain-of-thought prompting

Closest papers by meaning

• ReFT: Representation Finetuning for Language Models (NeurIPS 2024 · similarity 0.90)
• Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning (NeurIPS 2026 · similarity 0.86)
• Improving Reasoning Performance in Large Language Models via Representation Engineering (ICLR 2025 · similarity 0.86)
• Advancing Mathematical Reasoning in Language Models: The Impact of Problem-Solving Data, Data Synthesis Methods, and Training Stages (ICLR 2025 · similarity 0.86)
• Selection, Reflection and Self-Refinement: Revisit Reasoning Tasks via a Causal Lens (ICLR 2026 · similarity 0.85)
• DenseSteer: Steering Small Language Models towards Dense Math Reasoning (ICML 2026 · similarity 0.85)