Research map / Vision-language and multimodal / Multimodal LLMs

Breaking the SFT Plateau: Multimodal Structured Reinforcement Learning for Chart-to-Code Generation

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that SFT on chart-to-code data plateaus at about 2M samples and proposes MSRL, a two-stage GRPO training with rule-based textual rewards followed by a VLM-judged visual reward, reaching SOTA among open-source models on ChartMimic and ReachQA.

Problem

Supervised fine-tuning for chart-to-code generation saturates as data scales, and purely synthetic chart datasets are simplistic, so the performance ceiling of SFT limits structured visual-to-code generation.

Approach

A multi-granularity reward combining a normalized rule-based textual reward (data, chart type, layout, text, plus execution) with a visual reward from Qwen2.5-VL-72B judging rendered charts against ground truth, applied via a two-stage GRPO curriculum starting from a saturated SFT model.

Key results

• MSRL reaches 96.5 execution rate, 78.6 low-level and 83.8 high-level on ChartMimic Direct Mimic, outperforming all open-source models including ChartCoder (91.4/72.5/74.0).
• On ReachQA, MSRL achieves 98.2 execution rate and 89.9 high-level score, versus 83.8 execution and 69.4 high-level for ChartCoder.
• SFT performance saturates beyond about 2M samples; RL on the saturated SFT model lifts ChartMimic high-level from 77.6 to 82.7 with textual rewards only.

Surprising findings

• Textual-reward-only RL on a fresh (non-SFT) baseline yields lower low-level and high-level scores than SFT alone, despite a higher execution rate.
• DINO-based or InternViT visual evaluators give far smaller gains than a VLM judge, with DINO-L showing larger score discrepancies versus human ratings.

Limitations

• Visual reward relies on a 72B VLM judge, which is computationally expensive (1344 GPU hours for visual-only RL).
• Generalization tested only on small Seaborn and Plotly sets, with out-of-domain Matplotlib-trained model.
• Gap to GPT-4o remains on some low-level metrics such as text and type on ChartMimic.

Experimental setup

Datasets: arXiv table-derived chart-code corpus (3M) ChartCoder dataset (comparison)

Benchmarks: ChartMimic ReachQA Seaborn test set (107 images) Plotly test set (150 images)

Metrics: execution rate low-level score high-level score

Baselines: GPT-4o GPT-4V Claude-3-opus Qwen2.5-VL-7B Qwen2-VL-72B ChartCoder Chart2Code DPO

Builds on: GRPO MM-EUREKA LLaMA-Factory ChartMimic Chart2Code

Closest papers by meaning

• SketchVL: Policy Optimization via Fine-Grained Credit Assignment for Chart Understanding and More (CVPR 2026 · similarity 0.90)
• GoT-R1: Unleashing Reasoning Capability of Autoregressive Visual Generation with Reinforcement Learning (ICLR 2026 · similarity 0.89)
• ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generation (ICLR 2025 · similarity 0.89)
• Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.89)
• Thinking with Programming Vision: Towards a Unified View for Thinking with Images (CVPR 2026 · similarity 0.88)
• Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning (NeurIPS 2025 · similarity 0.88)