Research map / Large language models / Reasoning and chain of thought

SIM-CoT: Supervised Implicit Chain-of-Thought

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SIM-CoT, a plug-and-play training-time auxiliary decoder that supervises each implicit CoT latent with its corresponding explicit reasoning step, stabilizing latent training and improving accuracy of Coconut and CODI while adding no inference cost.

Problem

Implicit chain-of-thought methods lag behind explicit CoT, and scaling the number of latent tokens causes training instability and collapse (latents become homogeneous and drift from the vocabulary embedding space).

Approach

Add a training-only decoder that generates the k-th textual reasoning step from the k-th implicit latent, giving step-level supervision to each latent; the decoder is discarded at inference, and the same decoder doubles as a per-step interpretability probe.

Key results

• On GPT-2, SIM-CoT lifts Coconut from 36.6% to 44.8% on GSM8k-Aug (+8.2) and surpasses SFT-CoT (42.7%) by 2.1 points.
• On LLaMA-3.2 1B, SIM-CoT on CODI reaches 56.1% in-domain vs 52.7% for CODI (+3.4), with out-of-domain average 56.8%.
• On LLaMA-3.1 8B, SIM-CoT on CODI improves in-domain accuracy from 61.1% to 64.1% (+3.0) with unchanged token count (7.2).

Surprising findings

• Larger auxiliary decoders (3B or 8B) slightly reduce accuracy compared with a 1B decoder matching the backbone family.
• Implicit CoT with SIM-CoT on GPT-2 surpasses the explicit SFT-CoT baseline in-domain, which prior implicit methods did not achieve.

Limitations

• Evaluated only on arithmetic math word problems (GSM8k-Aug family) and LLaMA/GPT-2 backbones.
• Gap to explicit CoT remains on larger models such as LLaMA-3.2 3B and 8B in some metrics; gains on LLaMA-8B are modest on OOD tasks.
• Decoder size and alignment with backbone matter; larger decoders degraded performance.

Experimental setup

Datasets: GSM8K-Aug GSM8K

Benchmarks: GSM8k-Aug GSM-Hard MultiArith SVAMP

Metrics: accuracy number of tokens

Baselines: SFT-CoT No-CoT iCoT Coconut CODI Soft Thinking

Builds on: Coconut CODI iCoT Soft Thinking

Closest papers by meaning

• SemCoT: Accelerating Chain-of-Thought Reasoning through Semantically-Aligned Implicit Tokens (NeurIPS 2025 · similarity 0.92)
• SCOUT: Teaching Pre-trained Language Models to Enhance Reasoning via Flow Chain-of-Thought (NeurIPS 2025 · similarity 0.92)
• CoT Vectors: Transferring and Probing the Reasoning Mechanisms of LLMs (ICLR 2026 · similarity 0.91)
• Facts Don't Speak Louder than Words: The Behavioral Essence of Long-CoT Distillation (NeurIPS 2026 · similarity 0.91)
• L2V-CoT: Cross-Modal Transfer of Chain-of-Thought Reasoning via Latent Intervention (AAAI 2026 · similarity 0.91)
• Markovian Transformers for Informative Language Modeling (ICLR 2026 · similarity 0.90)