Research map / Vision-language and multimodal / Multimodal LLMs
FlowGen: Synthesizing Diverse Flowcharts to Enhance and Benchmark MLLM Reasoning
Kaiwen Shi, Sichen Liu, Ziyue Lin, Hangrui Guo, Gong Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FlowGen, a flowchart synthesis framework with fine-grained control over graph complexity and rendering style, used to train and benchmark multimodal LLM flowchart reasoning.
Problem
Interpreting flowcharts is hard due to structural complexity and visual diversity, and existing flowchart datasets lack fine-grained control over graph complexity and rendering style, limiting training and testing of multimodal LLMs.
Approach
A controllable flowchart generator that synthesizes diverse flowcharts with explicit control over graph complexity and rendering style, producing data both to enhance and to benchmark MLLM flowchart reasoning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on FlowGen synthetic flowcharts | Qwen2.5-VL-3B · FlowVQA parsing | strict F1: 12.7 -> 51.3 | Improved |
| fine-tune on FlowGen synthetic flowcharts | Qwen2.5-VL-7B · FlowLearn parsing | strict F1: 43.2 -> 60.1 | Improved |
| fine-tune on FlowGen synthetic flowcharts | Llava-V1.6-Mistral-7B · parsing benchmarks (FlowVQA) | strict F1: 2.6 -> 4.9 | No clear effect |
| fine-tune on FlowGen synthetic flowcharts | InternVL3-2B · FlowLearn parsing | strict F1: 21.8 -> 13.2 | Hurt |
| mix FlowGen parsing data into QA SFT | Qwen2.5-VL-7B · FlowLearn QA | accuracy: 85.3 -> 89.1 | Improved |
| mix FlowGen parsing data into QA SFT | Qwen2.5-VL-3B · FlowLearn QA | accuracy: 85.1 -> 84.7 | No clear effect |
| mix FlowGen parsing data into MMMU SFT | Qwen2.5-VL-7B · MMMU | accuracy: 62.6 -> 65.3 | Improved |
| prompt QA with triplets extracted by FlowGen-tuned Qwen2.5-VL-7B instead of self-extracted | Qwen2.5-VL-3B · FlowLearn QA | accuracy: 77.1 -> 88.2 | Improved |
| add FlowGen-extracted triplets to prompt | Qwen2.5-VL-7B · FlowLearn QA | accuracy: 71.1 -> 68.6 | Hurt |
| train on multiple renderers instead of only Mermaid (ablation) | FlowGen SFT (Qwen2.5-VL-7B) · CBD / FC B parsing | strict F1: 53.3 -> 55.4 (CBD); 38.1 -> 40.8 (FC B) | Improved |
| train on multiple renderers instead of only Mermaid (ablation) | FlowGen SFT (Qwen2.5-VL-7B) · FlowVQA / FlowLearn parsing | strict F1: 76.2 (Mermaid only) vs 70.9 (FlowVQA) | Hurt |
| add nested subgraphs to training data (ablation) | FlowGen SFT (Qwen2.5-VL-7B) · hdBPMN parsing | strict F1: 16.6 -> 18.3 | Improved |
Closest papers by meaning
- ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- Hint2Gen: Bridging Understanding and Generation via Code-structured Hints (CVPR 2026 · similarity 0.86)
- From Words to Structured Visuals: A Benchmark and Framework for Text-to-Diagram Generation and Editing (CVPR 2025 · similarity 0.86)
- Synthesize Step-by-Step: Tools Templates and LLMs as Data Generators for Reasoning-Based Chart VQA (CVPR 2024 · similarity 0.86)
- EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding (AAAI 2025 · similarity 0.85)
- CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
