Research map / Large language models / Reasoning and chain of thought
CATS: Category-Aware Token-level Steering for Training-Free Redundancy Reduction in Large Reasoning Models
Mengfei Zhang, Zhenglin Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CATS, a training-free token-level steering method that categorizes redundancy in large reasoning model chain-of-thought by whether it harms the final answer, to reduce reasoning redundancy.
Problem
Large reasoning models produce excessive redundancy in chain-of-thought reasoning, which lowers inference efficiency and raises compute cost.
Approach
Taxonomizes LRM redundancy into destructive and non-destructive (benign) categories based on whether it affects the final answer, and applies category-aware token-level steering to suppress redundancy without training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply CATS category-aware steering | DeepSeek-R1-Distill-Qwen-7B · MATH-500 | token length / accuracy: 3817.48 -> 2855.45 tokens; 92.96 -> 93.20 | Improved |
| apply CATS category-aware steering | DeepSeek-R1-Distill-Qwen-1.5B · MATH-500 | accuracy: 83.21 -> 82.01 with length 4528.31 -> 3577.36 | Mixed |
| apply CATS category-aware steering | DeepSeek-R1-Distill-Llama-8B · AIME 2024 | accuracy: 44.80 -> 46.14 | Improved |
| use six category vectors instead of one generic longest-minus-concise vector (ablation) | CATS (Qwen-7B) · AIME 2024 | accuracy: 48.66 -> 52.32 | Improved |
| use six category vectors instead of one generic vector (ablation) | CATS (Qwen-1.5B) · MATH-500 | accuracy: 79.55 -> 82.01 | Improved |
| apply generic intervention (single vector) (ablation) | Qwen-1.5B · MATH-500 | accuracy: 83.21 -> 79.55 | Hurt |
| apply CATS to reduce logical drift | DeepSeek-R1-Distill-Qwen-1.5B · Logical Drift occurrence rate | occurrence rate: 39.09 -> 31.27 | Improved |
| include the Logical Drift intervention vector (ablation) | CATS · average over benchmarks | accuracy / length: accuracy -4.2% and length +2.0% when removed | Improved |
| include the Repetition intervention vector (ablation) | CATS · average over benchmarks | length / accuracy: length +4.3% when removed; accuracy change less than 1.0% | Improved |
Closest papers by meaning
- Efficient Reasoning with Balanced Thinking (ICLR 2026 · similarity 0.88)
- Efficient Reasoning via Constrained Optimization in Latent Space (NeurIPS 2026 · similarity 0.88)
- R-KV: Redundancy-aware KV Cache Compression for Reasoning Models (NeurIPS 2025 · similarity 0.87)
- Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoring (NeurIPS 2026 · similarity 0.87)
- How Far Are We from Optimal Reasoning Efficiency? (NeurIPS 2025 · similarity 0.87)
- Short Chains, Deep Thoughts: Balancing Reasoning Efficiency and Intra-Segment Capability via Split-Merge Optimization (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
