Research map / Large language models / Reasoning and chain of thought

Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Landscape of Thoughts (LoT), a t-SNE-based visualization of LLM reasoning trajectories on multiple-choice data using perplexity-based state-to-choice distance features, plus consistency/uncertainty metrics and a lightweight random-forest verifier that improves weighted voting.

Problem

LLM step-by-step reasoning behavior is poorly understood; manual reading of trajectories does not scale or aggregate across datasets, so automated, objective analysis tools are needed.

Approach

Represent each intermediate reasoning state by its perplexity-based distances to all answer choices (computed with the same LLM), project these state features with t-SNE into density landscapes colored by correctness, and reuse the state features as inputs to a lightweight random-forest verifier for weighted majority voting.

Key results

• The LoT verifier improves over unweighted voting across models and decoding methods, with smaller 1B and 3B models gaining most with 10 trajectories.
• With up to 50 trajectories, verifier-based voting reaches over 65% accuracy while the unweighted baseline saturates around 30%.
• A verifier trained on AQuA improves StrategyQA accuracy by 4.5%; a verifier trained on the 70B model improves 3B model accuracy by 5.5%.

Surprising findings

• Incorrect trajectories often converge faster to wrong answers than correct trajectories converge to right answers within the same model and method.
• Uncertainty and perplexity generally increase with reasoning depth, even as larger models show higher consistency.

Limitations

• Restricted to multiple-choice datasets and to open-weight models that expose likelihoods; closed-source LLMs excluded.
• Analyses in main text use small samples (50 problems per dataset).
• Verifier transferability across datasets and models is inconsistent; some transfers do not help.

Experimental setup

Datasets: AQuA MMLU CommonsenseQA StrategyQA

Benchmarks: AQuA MMLU CommonsenseQA StrategyQA

Metrics: accuracy consistency uncertainty perplexity voting accuracy

Baselines: unweighted majority voting LLM-based verifiers

Builds on: t-SNE chain-of-thought tree-of-thought least-to-most prompting

Closest papers by meaning

• Probing the Trajectories of Reasoning Traces in Large Language Models (NeurIPS 2026 · similarity 0.89)
• Imagine While Reasoning in Space: Multimodal Visualization-of-Thought (ICML 2025 · similarity 0.88)
• MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.88)
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.88)
• Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark (AAAI 2024 · similarity 0.87)
• Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models (NeurIPS 2024 · similarity 0.87)