Research map / Large language models / Reasoning and chain of thought
Simple Test-Time Refinement for Plot-to-Code Generation via Visual-Code Diagnostics
Peiming Guo, Jinpeng Huang, Yangbin Sun, GaoXiong Cao, Ensheng Shi, Yuchi Ma, Yu Zhang, Meishan Zhang, Baotian Hu, Min zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a test-time refinement paradigm for plot-to-code generation in which a model generates code first, then iteratively locates and fixes errors using visual-code diagnostics.
Problem
Plot-to-code generation with MLLMs is typically evaluated with one-pass generation or iterative sampling filtered by verifiers, which does not mirror the human generate-then-debug programming workflow.
Approach
Replaces one-pass or verifier-filtered sampling with a test-time refinement loop that mimics human workflow: generate code, locate errors by comparing rendered output with the target plot, and fix them iteratively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual-code diagnostic verifier (RL-trained 2B) to guide patch refinement | Qwen3-VL-8B generator · ChartMimic | code execution success E: 77.8 -> 90.0 | Improved |
| same, visual score L | Qwen3-VL-8B generator · ChartMimic | visual score L: 66.2 -> 73.4 | Improved |
| same, GPT-4o generator | GPT-4o generator · ChartMimic | visual score L: 81.6 -> 87.6 | Improved |
| add RL stage after SFT for verifier (ablation) | Ours verifier · ChartMimic (Qwen3-VL-8B) | execution success E: 86.3 -> 90.0 | Improved |
| train verifier on diagnostic data vs untrained 32B verifier (ablation) | Qwen3-VL-8B generator · ChartMimic | visual score L: 70.9 (untrained 32B) -> 73.4 | Improved |
| add code localization signal to visual error type (ablation) | Verifier · ChartMimic (Qwen3-VL-8B) | visual score L: 70.7 -> 73.4 | Improved |
| add visual error type to code localization (ablation) | Verifier · ChartMimic (Qwen3-VL-8B) | visual score L: 72.2 -> 73.4 | Improved |
| use Jaccard reward instead of exact match (ablation) | Verifier RL · ChartMimic (Qwen3-VL-8B) | visual score L: 71.9 -> 73.4 | Improved |
| use Jaccard reward instead of set F1 (ablation) | Verifier RL · ChartMimic (Qwen3-VL-8B) | visual score L: 73.4 vs 73.4 | No clear effect |
| use diagnostic refinement instead of multi-agent refinement (token cost) | GPT-4o generator · Inference tokens | tokens: 11264.0 -> 1859.3 | Improved |
| apply diagnostic refinement to specialized ChartMaster-7B generator | ChartMaster-7B · ChartMimic | visual score L: 79.4 -> 63.4 | Hurt |
| Best-of-N sampling | Qwen3-VL-8B generator · ChartMimic | visual score L: 66.2 -> 66.9 | Improved |
Closest papers by meaning
- JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence (ICLR 2026 · similarity 0.87)
- PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning (NeurIPS 2026 · similarity 0.86)
- VCG-Bench: Towards A Unified Visual-Centric Benchmark for Structured Generation and Editing (ICML 2026 · similarity 0.86)
- VisCodex: Unified Multimodal Code Generation via Merging Vision and Coding Models (ICLR 2026 · similarity 0.85)
- Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation (ICLR 2026 · similarity 0.85)
- Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
