Research map / Large language models / Reasoning and chain of thought
Enhancing LLM Reasoning via Vision-Augmented Prompting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Vision-Augmented Prompting (VAP), a training-free framework where an MLLM plans, iteratively draws images with external tools (Matplotlib, Turtle, DALL·E 3) interleaved with textual thoughts, and then answers using self-alignment-checked images.
Problem
LLM reasoning frameworks such as CoT and its variants are text-only, which limits performance on problems with visual and spatial clues (geometry, grid puzzles, time series, routing).
Approach
Synthesize an image from the textual problem using external drawing tools, iteratively update it with a chain of thoughts in both image and text modalities, and gate the image with a self-alignment check before final answer generation.
Key results
- On Geometry Intersection Counting, VAP reaches 16.5% accuracy versus 11.5% for CoT-SC (k=20) and 8.5% for standard prompting.
- On Sudoku, VAP achieves a 35.5% correct rate versus 22.6% for ToT.
- On TSP with N=10, VAP gives a 6.0% optimality gap versus 10.4% for CoT-SC; VAP+LLMTime is best on time series prediction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VAP instead of G-LLaVA | Geometry Intersection Counting · Geometry Intersection Counting | accuracy: 14.0% -> 16.5% | Helped |
| Use VAP instead of ToT | Sudoku Puzzle · Sudoku Puzzle | correct rate: 22.6% -> 35.5% | Helped |
| Combine VAP with LLMTime preprocessing | Time Series Prediction · Time Series Prediction | MAE: 617 -> 556 | Helped |
| Remove high-level planning (ablation) | VAP · Geometry Intersection Counting | accuracy: 12.0% -> 16.5% | Helped |
| Remove iterative reasoning (ablation) | VAP · Geometry Intersection Counting | accuracy: 14.5% -> 16.5% | Helped |
| Remove self-alignment (ablation) | VAP · Geometry Intersection Counting | accuracy: 16.0% -> 16.5% | Helped |
| Remove high-level planning (ablation) | VAP · Sudoku Puzzle | correct rate: 24.7% -> 35.5% | Helped |
| Remove iterative reasoning (ablation) | VAP · Sudoku Puzzle | correct rate: 19.9% -> 35.5% | Helped |
| Remove self-alignment (ablation) | VAP · Sudoku Puzzle | correct rate: 25.1% -> 35.5% | Helped |
| Remove high-level planning (ablation) | VAP · Time Series Prediction | MSE: 578 -> 556 | Helped |
| Remove iterative reasoning (ablation) | VAP · Time Series Prediction | MSE: 582 -> 556 | Helped |
| Remove self-alignment (ablation) | VAP · Time Series Prediction | MSE: 556 -> 556 | No clear effect |
Surprising findings
- Synthesized images improved dedicated geometry solvers Inter-GPS (6.0% to 17.0%) and G-LLaVA (14.0% to 20.5%).
- Removing the self-alignment step did not change time series performance, since the drawing module had no errors there.
Limitations
- Performance gap remains between VAP and task-specific approaches.
- Black-box behaviour; interpretability not established.
- Relies on MLLM instruction following, which sometimes fails (e.g., repeated drawing), and evaluation uses small, sampled test sets.
Experimental setup
Datasets: BIG-bench Geometry Intersection Counting (200 sampled) BIG-bench Sudoku generator (150 puzzles) Darts time series datasets (8 series) Randomly generated Euclidean TSP instances (N=10, 20; 100 each)
Benchmarks: BIG-bench Geometry Intersection Counting BIG-bench Sudoku Darts time series Euclidean TSP N=10/20
Metrics: accuracy correct rate collision rate MAE tour length gap to optimal integrity rate
Baselines: Standard prompting CoT CoT-SC Tree of Thoughts Inter-GPS G-LLaVA LLMTime SMA Gurobi Nearest neighbour Fastest insertion Random
Builds on: Chain-of-Thought Self-consistency Tree of Thoughts LLMTime
Closest papers by meaning
- Visual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning (AAAI 2024 · similarity 0.89)
- Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing (NeurIPS 2025 · similarity 0.89)
- Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models (NeurIPS 2024 · similarity 0.88)
- When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.88)
- Imagine While Reasoning in Space: Multimodal Visualization-of-Thought (ICML 2025 · similarity 0.88)
- Revisiting the Necessity of Lengthy Chain-of-Thought in Vision-centric Reasoning Generalization (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.