AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

DiagramGPT-Llama3:Enabling Editable, High-Fidelity Diagram Generation with Vision Large Language Models

Yongyuan Chen, Minjie Hong, Boxi Wu, Xicheng Han

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes generating editable, high-fidelity diagrams (e.g., draw.io/XML-style) from text or raster images, introducing diagram restoration and style transfer tasks, using a vision LLM approach named DiagramGPT-Llama3.

Problem

Existing diagram generation from natural language often lacks editable output that supports user-friendly editing such as drag-and-drop.

Approach

A new task of generating editable, high-fidelity diagrams from either text or raster images, plus diagram restoration and style transfer tasks, addressed with a vision large language model built on Llama 3.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add FABS to base model (ablation)DiagramGPT base model · Diagram-mxGraph testOverall score: 66.854 -> 66.988Improved
add CAAL to base model (ablation)DiagramGPT base model · Diagram-mxGraph testOverall score: 66.854 -> 66.966Improved
add both FABS and CAAL (ablation)DiagramGPT base model · Diagram-mxGraph testOverall score: 66.854 -> 67.188Improved
add CAAL to base model (ablation)DiagramGPT base model · Diagram-mxGraph testCaptioning CIDEr: 4.54 -> 9.16Improved
add FABS alone to base model (ablation)DiagramGPT base model · Diagram-mxGraph testCaptioning CIDEr: 4.54 -> 3.44Hurt
add CAAL to base model (ablation)DiagramGPT base model · Diagram-mxGraph test, Task 3TaskScore: 52.190 -> 52.887Improved
add FABS to base model (ablation)DiagramGPT base model · Diagram-mxGraph test, Task 1TaskScore: 95.780 -> 95.809No clear effect
add CAAL to base model (ablation)DiagramGPT base model · Diagram-mxGraph test, Task 1TaskScore: 95.780 -> 95.094Hurt
add FABS and CAAL to base model (ablation)DiagramGPT base model · Diagram-mxGraph test, Task 1TaskScore: 95.780 -> 95.298Hurt
fine-tune DiagramGPT instead of few-shot ChatGPT-4o-minidiagram generation · Diagram-mxGraph test, Task 3TaskScore: 55.006 -> 52.739Hurt
use DiagramGPT instead of one-shot ChatGPT-4o-minidiagram generation · human evaluation Task 1 accuracyaccuracy (0-100): 59.670 -> 73.977Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy