Research map / Large language models / Reasoning and chain of thought

Reasoning Models Can be Accurately Pruned Via Chain-of-Thought Reconstruction

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reasoning-Aware Compression (RAC), which adds on-policy chain-of-thought activations to the calibration set of post-training pruning methods (SparseGPT, WANDA, ALPS) so one-shot pruning better preserves reasoning accuracy and runtime.

Problem

One-shot pruning of reasoning LLMs with prompt- or C4-based calibration causes large accuracy loss and, at high sparsity, longer and more rambling chains of thought that slow inference.

Approach

Augment layer-wise pruning calibration activations with activations from the dense model's own self-generated (on-policy) CoT traces, so the reconstruction objective matches decode-time activation distribution; it is a drop-in change to existing pruners.

Key results

• On DeepSeek-R1-Distill-Qwen-1.5B at 50% sparsity on MATH-500, RAC reaches 0.664 accuracy versus 0.496 for prompt-only and 0.356 for C4.
• On Qwen3-8B at 50% sparsity on MATH-500, RAC achieves 0.862 accuracy versus 0.564 (C4) and 0.470 (prompt-only), reducing evaluation time from about 259 minutes to 17.1 minutes.
• On DeepSeek-R1-Distill-Qwen-7B at 50% sparsity, RAC keeps 0.900 accuracy versus 0.936 dense and cuts runtime from 135.0 to 35.3 minutes versus C4.

Surprising findings

• Heavy pruning with standard calibration can slow inference because the model generates longer, lower-accuracy chains of thought.
• Reducing the max decode budget to 4,096 tokens lets the RAC-pruned 7B model slightly outperform the dense model (0.836 vs 0.824).

Limitations

• Does not directly address increased inference length; relies on reducing max decode budget to mitigate.
• Collecting on-policy CoT activations increases calibration cost, especially for large decode budgets.
• Off-policy traces (from a larger model) reduce accuracy at high sparsity, indicating dependence on same-model traces.

Experimental setup

Datasets: C4 Open-R1-Math-220k Codeforces

Benchmarks: MATH-500 AIME-25 LiveCodeBench

Metrics: acc@1:1 codegen_pass@1:16 standard error runtime (minutes) throughput (tok/s)

Baselines: SparseGPT with C4 calibration SparseGPT with prompt-only calibration WANDA ALPS

Builds on: SparseGPT WANDA ALPS GRPO on-policy distillation

Closest papers by meaning

• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression (ICML 2026 · similarity 0.91)
• Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy (ICLR 2026 · similarity 0.90)
• When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models (ICLR 2026 · similarity 0.90)
• OS-Pruner: Pruning Chains-of-Thought of Reasoning Models via Optimal Stopping (NeurIPS 2026 · similarity 0.90)
• Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains (NeurIPS 2025 · similarity 0.89)
• Reasoning Path Compression: Compressing Generation Trajectories for Efficient LLM Reasoning (NeurIPS 2025 · similarity 0.89)