AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

DaVinci: Reinforcing Visual-Structural Syntax in MLLMs for Generalized Scientific Diagram Parsing

Xingchen ZENG, Zhewei Su, Hengming Zhang, Juyong Jiang, Jiazhi Xia, Wei Zeng

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DaVinci, an MLLM for parsing raster scientific diagrams into structured representations, trained with a two-stage supervised-then-reinforcement framework.

Problem

Parsing raster-based scientific diagrams into structured, editable representations is hard for existing MLLMs due to diverse visual primitives, complex layouts, and strict syntax requirements.

Approach

A two-stage training framework for diagram parsing: supervised learning on visual primitives followed by reinforcement learning to reinforce visual-structural syntax in the MLLM.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
reorder TikZ code into a logical drawing order (ablation)Qwen2.5-VL-7B SFT · DaTikZv3Pass@1 compile rate: 69.74 -> 78.78Improved
add LLM-written planning comments to TikZ training code (ablation)Qwen2.5-VL-7B SFT · DaTikZv3Pass@1 compile rate: 78.78 -> 84.50Improved
add GRPO RL with hybrid reward after SFTDaVinci-SFT-7B · DaTikZv3Pass@1 compile rate: 84.50 -> 97.60Improved
add GRPO RL after SFTDaVinci-SFT-7B · DaTikZv3cBLEU: 7.52 -> 6.57Hurt
add spatio-textual reward Rtext (ablation)DaVinci RL · DaTikZv3Textual match score: 37.23 -> 41.58Improved
add geometric reward Rgeom on top of Rtext (ablation)DaVinci RL · DaTikZv3Geometry match score: 42.12 -> 44.10Improved
add SSIM reward to base image reward (ablation)DaVinci RL · DaTikZv3Textual/Geometry match: 37.23 -> 37.45 (textual), 41.44 -> 41.87 (geometry)No clear effect
enable explicit thinking mode (ablation)GLM-4.5V · DaTikZv3Pass@1 compile rate: 67.90 -> 62.92Hurt
enable explicit thinking mode (ablation)Claude-Sonnet-4 · DaTikZv3Pass@1 compile rate: 84.87 -> 86.90No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy