Research map / Large language models / Reasoning and chain of thought

ImgCoT: Compressing Long Chain of Thought into Compact Visual Tokens for Efficient Reasoning of Large Language Model

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Chain-of-thought training and analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ImgCoT, which compresses textual chain-of-thought by rendering it into images and autoencoding the visual CoT into 8 discrete latent tokens (TiTok-based), plus a hybrid L-ImgCoT that keeps low-likelihood textual steps.

Problem

Latent CoT compression with text-reconstruction autoencoders forces latent tokens to encode surface linguistic form rather than reasoning structure, limiting logical abstraction and reasoning accuracy while reasoning cost remains high.

Approach

Replace the textual CoT reconstruction target with a rendered visual CoT image (segments in boxes with arrows), so a 1D visual tokenizer's latent tokens capture spatial/global reasoning layout; L-ImgCoT additionally keeps textual steps whose average log-likelihood is below a threshold derived from a general corpus.

Key results

• On Qwen2.5-1.5B-Instruct, ImgCoT reaches 38.7 on GSM with 8 tokens versus 44.1 for Full-CoT with 126.6 tokens, and L-ImgCoT reaches 45.2 on GSM.
• L-ImgCoT consistently outperforms Full-CoT across all settings while reducing inference cost by roughly 30%, per the authors.
• Visual latent tokens outperform text-reconstruction latent tokens (w/ textual tokens) across the tested settings, with out-of-domain gains on SVAMP, SingleEq and MultiArith for Qwen2.5-0.5B.

Surprising findings

• Performance of latent reasoning degrades when the number of latent tokens becomes very large, which the authors attribute to underfitting.

Limitations

• Full fine-tuning only for sub-1B models; LoRA used for larger models due to compute constraints, and some baseline results (CODI on Llama) are missing.
• Performance degrades with many latent tokens, attributed to underfitting with fixed training data.
• Visual latent tokens may blur fine-grained domain-specific reasoning details, requiring the hybrid L-ImgCoT variant.

Experimental setup

Datasets: MathPile MetaMathQA GSM8K MATH GPQA-Extended ProsQA

Benchmarks: GSM8K MATH GPQA-Extended ProsQA Gaokao-Math-2023 SVAMP SingleEq MultiArith

Metrics: accuracy number of reasoning tokens

Baselines: Full-CoT Coconut ICoT CODI CoLaR

Builds on: TiTok Glyph Coconut CODI CoLaR

Closest papers by meaning

• SIM-CoT: Supervised Implicit Chain-of-Thought (ICLR 2026 · similarity 0.90)
• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.90)
• Latent Visual Reasoning (ICLR 2026 · similarity 0.90)
• Long Chain-of-Thought Reasoning Across Languages (ICLR 2026 · similarity 0.90)
• Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (NeurIPS 2024 · similarity 0.89)
• Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression (ICML 2026 · similarity 0.89)