Research map / Large language models / Reasoning and chain of thought

Efficient Reasoning with Hidden Thinking

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Heima compresses multi-stage visual CoTs in an MLLM into single hidden thinking tokens via progressive distillation, and uses LLM interpreters to reconstruct the reasoning text from those tokens.

Problem

Textual chain-of-thought in multimodal LLMs is verbose and expensive at inference; latent CoT compression has mostly been tested on small text-only math models, not general multimodal reasoning.

Approach

Replace each CoT stage (summary, caption, reasoning) with a dedicated special thinking token, distilled progressively one stage at a time, plus a recovery stage; then train separate text-only LLM interpreters that map the token's last hidden state back to the stage's text, with an information-theoretic argument that compression cannot increase task information beyond the full CoT.

Key results

• Heima averages 58.0% accuracy across six benchmarks versus 61.1% for LLaVA-CoT and 52.1% for Llama3.2-11B-Vision, while generating roughly 12-17 tokens per answer versus about 155-230 for LLaVA-CoT.
• On MMBench, Heima (72.8%) exceeds LLaVA-CoT (70.7%) while using about 13 tokens instead of 155.
• Progressive distillation outperforms one-shot distillation (58.0 vs 56.3 average) and the recovery stage adds accuracy (58.0 vs 56.6).

Surprising findings

• Heima reaches higher accuracy than LLaVA-CoT on MMBench and AI2D despite using far fewer tokens.
• Ablation over retention ratio shows accuracy fluctuates without a clear trend, and at 70% retention the token count exceeds the baseline.

Limitations

• Accuracy is below LLaVA-CoT on average (58.0 vs 61.1) and drops notably on MMVet and MathVista.
• Interpreter reconstruction quality is only moderate (BLEU-4 around 11-16), so the claim of a minimal information gap rests on proxy metrics and GPT-4o judging.
• Evaluated only on Llama-3.2-11B and LLaVA-Next-Vicuna-7B with a single training dataset; the authors plan larger models as future work.

Experimental setup

Datasets: LLaVA-CoT-100k

Benchmarks: MMStar MMBench V1.1 MMVet MathVista AI2D HallusionBench

Metrics: accuracy number of generated tokens BLEU-4 METEOR ROUGE-L BERTScore GPT-4o similarity rank (1-5)

Baselines: Llama3.2-11B-Vision-Instruct LLaVA-CoT LLaVA-Next-Vicuna-7B (CoT) Heima w/o progressive Heima w/o recover

Builds on: LLaVA-CoT Chain-of-Thought prompting Coconut

Closest papers by meaning

• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression (ICML 2026 · similarity 0.91)
• Making Slow Thinking Faster: Compressing LLM Chain-of-Thought via Step Entropy (ICLR 2026 · similarity 0.91)
• Deep Hidden Cognition Facilitates Reliable Chain-of-Thought Reasoning (AAAI 2026 · similarity 0.90)
• Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization (AAAI 2026 · similarity 0.90)
• C3oT: Generating Shorter Chain-of-Thought Without Compromising Effectiveness (AAAI 2025 · similarity 0.90)
• A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings (NeurIPS 2025 · similarity 0.90)