Research map / Multi-agent systems and games / LLM agents

Tracing the Cascade: A Topology-Aware Evaluation Framework for Scientific Agent Hallucinations

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SCHEMA, a topology-aware, evidence-grounded framework that builds biomedical concept graphs, synthesizes graph-grounded tasks, and scores agent hallucinations with a degree-weighted severity metric plus counterfactual attribution.

Problem

Existing hallucination benchmarks evaluate facts in isolation with uniform accuracy metrics, missing how an erroneous claim on a densely connected scientific concept cascades through multi-step agent reasoning.

Approach

Weights per-claim hallucination severity by node degree in a scientific knowledge graph built from benchmark seeds and literature, and pairs it with a multi-agent counterfactual intervention (abduction, intervention, prediction with frozen context) that certifies root-cause step for failed trajectories.

Key results

• GPT-5.4-mini achieves the lowest HS^w on PathVQA-Enhanced (0.287) and lowest HS on both subsets (0.179 Protein, 0.122 PathVQA).
• Gemini-3-Flash reaches the top Protein Domain accuracy of 0.747, while Doubao-Seed-2.0-Pro reaches 0.677 on PathVQA-Enhanced.
• Hallucination severity HS^w rises monotonically along the verifiability axis from 0.381 (identifier-grounded) to 0.436 (locative).

Surprising findings

• Final-answer accuracy is only weakly correlated with topology-weighted hallucination severity; models reach correct answers via flawed reasoning (e.g., DeepSeek-V4-Flash and GPT-5.4-mini tie on accuracy at 0.59 but differ in HS^w 0.362 vs 0.268).
• Six of ten models show a performance valley at Weak Anchoring, scoring lower than both Strong and Unanchored samples.

Limitations

• Evaluated only on two biomedical subdomains; generality to other sciences untested.
• Trajectory hallucination pipeline relies on LLM judge without large-scale expert validation.
• Counterfactual attribution is costly and applied only to a sampled subset of failures (141 trajectories from 346 questions).

Experimental setup

Datasets: ProteinLMBench PathVQA-Enhanced PubMed bioRxiv

Benchmarks: SCHEMA Protein Domain SCHEMA PathVQA-Enhanced

Metrics: accuracy WAcc hallucination rate (HR) hallucination severity (HS) topology-weighted hallucination severity (HS^w)

Baselines: GPT-4o GPT-5.4-mini Gemini-3-Flash Grok-4.1-Fast Kimi-K2.5 Qwen3.6-Plus Llama-4-Scout Doubao-Seed-2.0-Pro DeepSeek-V4-Flash Intern-S1-Pro

Builds on: ReAct AGDebugger AutoGen SapBERT

Closest papers by meaning

• Auditing Meta-Cognitive Hallucinations in Reasoning Large Language Models (NeurIPS 2025 · similarity 0.91)
• InEx: Hallucination Mitigation via Introspection and Cross-Modal Multi-Agent Collaboration (AAAI 2026 · similarity 0.89)
• Mitigating Hallucinations in Large Language Models via Causal Reasoning (AAAI 2026 · similarity 0.88)
• Multi-Agent Undercover Gaming: Hallucination Removal Through Counterfactual Test for Multimodal Reasoning (AAAI 2026 · similarity 0.88)
• Dual-Pathway Circuits of Object Hallucination in Vision-Language Models (NeurIPS 2026 · similarity 0.88)
• MIRAGE: Assessing Hallucination in Multimodal Reasoning Chains of MLLM (NeurIPS 2025 · similarity 0.87)