Research map / Multi-agent systems and games / LLM agents
DV-World: Benchmarking Data Visualization Agents in Real-World Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DV-World, a 260-task benchmark evaluating data visualization agents across native spreadsheet charting/fixing/dashboards (DV-Sheet), cross-framework chart evolution from reference images (DV-Evol), and clarification-based interactive visualization under ambiguous intent (DV-Inter), with hybrid rule-based and MLLM-as-judge scoring.
Problem
Existing data visualization benchmarks are confined to code sandboxes, creation-only one-shot tasks, and fully specified intents, failing to capture native spreadsheet workflows, iterative chart evolution, and ambiguous user requests.
Approach
A lifecycle benchmark spanning native spreadsheet object-model manipulation (create, fix, dashboard), cross-framework visualization evolution from reference images, and multi-turn clarification with a user simulator that has hidden true intents and reaction rules, scored by expert rubrics, table-value alignment, and an MLLM judge.
Key results
- Best DV-Sheet overall score is 40.48 (Gemini-3-Pro with DV-World-Agent); no model in DV-Sheet exceeds 40.5 while human reference is far higher.
- On DV-Evol, Gemini-3-Pro reaches the highest overall score of 51.44 versus human baseline around 84-88 on MS/TC.
- On DV-Inter, Grok-4 achieves the top score of 40.43, with ISR 79.57; judge-model agreement reaches weighted kappa 0.821 for Gemini-2.5-Flash against human 0.903 agreement.
Surprising findings
- Qwen3-8B scores only 6.91 on DV-Sheet dashboards, and many models fail badly on native spreadsheet creation.
- Performance drops as target lines of code grow, with D3.js the hardest framework for agents.
Limitations
- Ambiguity-driven user simulator relies on GPT-5-Mini, so simulator fidelity affects interaction scores.
- MLLM-judge scores depend on judge choice, with rankings only partially stable across judges.
- Benchmark is small (260 tasks) and human baseline covers only 10 evaluators on 10 tasks per domain.
Experimental setup
Datasets: Kaggle datasets ExcelForum MrExcel Chandoo.org Excelguru
Benchmarks: DV-World
Metrics: rubric score table coverage success rate interaction success rate MLLM-score
Baselines: SheetCopilot OpenHands DV-World-Agent
Builds on: ReAct OpenHands SheetCopilot
Closest papers by meaning
- CoDA: Agentic Systems for Collaborative Data Visualization (ICLR 2026 · similarity 0.85)
- VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries (NeurIPS 2026 · similarity 0.84)
- VisJudge-Bench: Aesthetics and Quality Assessment of Visualizations (ICLR 2026 · similarity 0.84)
- DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecycle (ICLR 2026 · similarity 0.83)
- PerfGuard: A Performance-Aware Agent for Visual Content Generation (ICLR 2026 · similarity 0.83)
- ShowTable: Unlocking Creative Table Visualization with Collaborative Reflection and Refinement (CVPR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.