Research map / Large language models / Reasoning and chain of thought
C3oT: Generating Shorter Chain-of-Thought Without Compromising Effectiveness
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes C3oT, which compresses long chain-of-thought rationales with GPT-4 and trains LLaMA-2-Chat models on both long and short CoT under distinct prompt-token conditions, so short CoT can be generated at inference with near-long-CoT accuracy.
Problem
Chain-of-thought rationales are much longer than the final answers, raising decoding cost and latency for real-world applications, while naively shortening CoT hurts LLM reasoning accuracy.
Approach
A GPT-4 compressor produces shorter CoT for each training rationale; the model is SFT-trained on both long and short versions distinguished by condition prompts (class-conditioned policy), then at inference it is prompted with the short-condition prompt to generate compressed reasoning.
Key results
- On GSM8K with 13B, C3oT reaches 47.10% accuracy vs 48.07% for Long CoT while compressing CoT by 57.78%.
- On ECQA with 7B, C3oT achieves 69.38% accuracy, above Long CoT (63.96%), with 51.55% compression.
- Implicit-CoT drops to 11.16% accuracy on GSM8K (7B), far below explicit methods.
- Short CoT trained without conditioning loses accuracy (31.01% vs 37.38% on GSM8K 7B).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-7B · GSM8K | accuracy: 37.38 -> 36.92 | Hurt |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-7B · MathQA | accuracy: 51.46 -> 50.35 | Hurt |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-7B · ECQA | accuracy: 63.96 -> 69.38 | Helped |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-7B · StrategyQA | accuracy: 69.66 -> 72.41 | Helped |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-13B · GSM8K | accuracy: 48.07 -> 47.10 | Hurt |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-13B · MathQA | accuracy: 56.21 -> 56.62 | Helped |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-13B · ECQA | accuracy: 68.92 -> 71.93 | Helped |
| Use conditioned C3oT instead of Long CoT | LLaMA-2-Chat-13B · StrategyQA | accuracy: 76.21 -> 76.55 | Helped |
| Add distinct conditions for longer and shorter CoT during training (ablation) | C3oT, LLaMA-2-Chat-7B · GSM8K | accuracy: 34.50 -> 36.92 | Helped |
| Add distinct conditions for longer and shorter CoT during training (ablation) | C3oT, LLaMA-2-Chat-7B · MathQA | accuracy: 48.07 -> 50.35 | Helped |
| Add distinct conditions for longer and shorter CoT during training (ablation) | C3oT, LLaMA-2-Chat-7B · ECQA | accuracy: 66.93 -> 69.38 | Helped |
| Add distinct conditions for longer and shorter CoT during training (ablation) | C3oT, LLaMA-2-Chat-7B · StrategyQA | accuracy: 70.00 -> 72.41 | Helped |
Surprising findings
- Simply training on GPT-4-compressed short CoT substantially reduces accuracy, yet conditioning on both lengths recovers near-long-CoT performance.
- C3oT outperforms Long CoT on ECQA and StrategyQA at both model sizes.
Limitations
- Compression with GPT-4 and over-compression degrades accuracy; performance depends on compressed CoT retaining key information.
- Evaluated only on LLaMA-2-Chat 7B/13B and four datasets; math tasks still lag Long CoT slightly.
- Relies on closed-source GPT-4 for compression, adding data-construction cost.
Experimental setup
Datasets: GSM8K MathQA ECQA StrategyQA
Benchmarks: GSM8K MathQA ECQA StrategyQA
Metrics: exact match accuracy compression rate
Baselines: Short CoT Long CoT Implicit-CoT w/o condition
Builds on: chain-of-thought Implicit-CoT OpenChat Orca 2
Closest papers by meaning
- Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression (ICML 2026 · similarity 0.93)
- Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMs (NeurIPS 2024 · similarity 0.91)
- Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization (AAAI 2026 · similarity 0.91)
- Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy (ICLR 2026 · similarity 0.91)
- Training Nonlinear Transformers for Chain-of-Thought Inference: A Theoretical Generalization Analysis (ICLR 2025 · similarity 0.90)
- Efficient Reasoning with Hidden Thinking (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.