Research map / Large language models / Language, safety and interpretability

nvBench 2.0: Resolving Ambiguity in Text-to-Visualization through Stepwise Reasoning

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Controlled text generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces nvBench 2.0, a benchmark of ambiguous text-to-visualization queries built by a reverse-generation ambiguity-injection pipeline, and Step-Text2Vis, a Qwen2.5-7B model trained with SFT cold-start plus step-wise DPO that reaches the best F1@3 and F1@5 on it.

Problem

Text-to-Visualization systems are evaluated under a single-correct-answer paradigm, so they cannot be tested on or trained for ambiguous natural language queries that map to multiple valid visualizations.

Approach

Generate ambiguous queries in reverse: start from unambiguous seed visualization trees, inject data-layer and visualization-layer ambiguities, enumerate all valid resolutions with an ASP solver, and produce step-wise disambiguation reasoning paths, then train with step-wise DPO on the first-error steps.

Key results

• Step-Text2Vis achieves the highest F1@3 (81.50%) and F1@5 (80.88%) on nvBench 2.0, versus 58.96% and 59.03% for GPT-4o.
• Step-Text2Vis reaches 99.20% precision at K=1 and 37.30% recall at K=1.
• Qwen2.5-7B-SFT reaches 75.79% F1@3, showing SFT alone already improves over prompting baselines of similar size.

Measured effects reported in the paper

Surprising findings

• Step prompting did not help all models: Claude-3.5-Haiku-Step and GPT-4o-mini-Step show mixed gains relative to direct prompting, and GPT-4o-Step gains are small.
• Fine-tuned SFT model trades precision for recall relative to prompting baselines.

Limitations

• Ambiguity levels are capped at k<=5 and ambiguity is synthetically injected rather than collected from real users.
• Query generation relies on GPT-4o and GPT-4o-mini, with about 5% of queries introducing unintended facts that needed manual correction.
• No error bars or statistical significance tests are reported.

Experimental setup

Datasets: nvBench 2.0 nvBench 1.0 BIRD

Benchmarks: nvBench 2.0

Metrics: Precision@K Recall@K F1@K

Baselines: GPT-4o-mini GPT-4o Claude-3.5-Haiku Qwen2.5-7B Qwen3-235B Qwen2.5-7B-SFT

Builds on: nvBench Step-DPO Chain-of-Thought prompting ScienceBenchmark

Closest papers by meaning

• VisInteract: Towards Dynamic Interactive Text-to-Visualization under Imperfect Queries (NeurIPS 2026 · similarity 0.89)
• Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models (ICLR 2026 · similarity 0.86)
• Multimodal DeepResearcher: Generating Text-Chart Interleaved Reports from Scratch with Agentic Framework (AAAI 2026 · similarity 0.86)
• ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.85)
• GraphVis: Boosting LLMs with Visual Knowledge Graph Integration (NeurIPS 2024 · similarity 0.85)
• Magnifying What Matters: Attention-Guided Adaptive Rendering for Visual Text Comprehension (NeurIPS 2026 · similarity 0.84)