Research map / Large language models / Retrieval and knowledge

VeriGraph: Towards Verifiable Data-Analytic Agents

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: Knowledge graph reasoning with LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VeriGraph, a neuro-symbolic data-analytic agent that builds an explicit heterogeneous evidence DAG (computational, grounding, derivational edges) during execution and trains Qwen3-8B with SFT cold-start plus graph-aware DAPO using a composite reward.

Problem

Outputs of LLM data-analytic agents are hard to verify because computations and semantic claims are entangled in linear text trajectories, making numerical results hard to reproduce and qualitative judgments hard to inspect.

Approach

Replace the linear thought-action-observation transcript with an incrementally built evidence DAG using bind (grounding), infer (derivation), and automatic computational tracing primitives inside the code action space, and optimize the policy with a composite reward covering execution success, derivation verification, and terminal subgraph judged quality.

Key results

• VeriGraph-8B achieves the highest Overall score (73.68) among evaluated baselines, matching Claude-4.5-Opus ReAct (73.22).
• VeriGraph-8B reaches 87.61% Grounding Rate, +14.04 over the strongest ReAct baseline.
• Removing trajectory SFT drops Overall by 35.90 points and TableBench from 73.58 to 6.22; outcome-only RL lowers Overall to 65.35 and GR to 76.46.

Surprising findings

• Outcome-only RL degrades both accuracy and grounding relative to SFT-only policy (65.35 vs 70.42 Overall).
• Structured graph-serialized answers receive lower LLM-judge Content/Format scores than free-form prose despite higher grounding.

Limitations

• Grounding Rate measures recoverability of claims from the exposed evidence artifact, not semantic truth of bind descriptions.
• Content/Format LLM-judge scores penalize structured outputs relative to direct-inference baselines.
• Graph-based grounding relies on LLM-based judges for derivation validity and evaluation.

Experimental setup

Datasets: TableBench InfiAgent-DABench DSBench DABstep

Benchmarks: TableBench InfiAgent-DABench DSBench DAB-Step Research

Metrics: accuracy Content score Format score Grounding Rate (GR) Overall score

Baselines: GPT-5.2 Gemini-2.5-Pro Claude-4.5-Sonnet Claude-4.5-Opus Qwen3-32B Qwen3-30B-A3B GPT-5.4 Qwen3-8B Qwen3-Coder-30B QwQ-32B DataMind DeepAnalyze

Builds on: ReAct DAPO CodeAct GRPO

Closest papers by meaning

• ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence (NeurIPS 2026 · similarity 0.87)
• Talk like a Graph: Encoding Graphs for Large Language Models (ICLR 2024 · similarity 0.86)
• Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning (ICLR 2024 · similarity 0.86)
• Heterogeneous Graph Reasoning for Fact Checking over Texts and Tables (AAAI 2024 · similarity 0.86)
• Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph (ICLR 2024 · similarity 0.85)
• $\texttt{G1}$: Teaching LLMs to Reason on Graphs with Reinforcement Learning (NeurIPS 2025 · similarity 0.85)