AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

HiddenPathQA: A Benchmark for Knowledge Graph Question Answering When Questions Hide Their Paths

Sumin Jo, Edward Choi

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: Knowledge graph reasoning with LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces HiddenPathQA, a KGQA benchmark whose questions omit lexical cues to the gold knowledge-graph path, so systems must infer multi-hop schema paths from implicit phrasing.

Problem

Existing KGQA evaluations use questions whose wording lexically exposes the gold KG path, whereas real users ask without knowing the schema, so current systems may be overestimated.

Approach

A benchmark where questions compress multi-hop KG paths into single natural phrases (e.g., "family history" for a 4-hop father/medical condition chain), testing path inference rather than lexical matching.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace hop-by-hop pruning with holistic chain selection (TieredHolistic)KGQA LLM pipeline · HiddenPathQA Pseudoword (GPT-5-mini)accuracy: 41.7 -> 59.6Improved
replace hop-by-hop pruning with holistic chain selection (FlatHolistic)KGQA LLM pipeline · HiddenPathQA Original (Qwen3-32B)accuracy: 69.1 -> 78.5Improved
remove entity names (pseudoword replacement)closed-book CoT / Zero-shot · HiddenPathQAaccuracy: 64.6 -> 25.3 (Qwen3-32B)Hurt
add KG retrieval to LLM (ToG, FiDeLiS, KARPA) on Original splitLLM · HiddenPathQA Originalaccuracy: 69.1 -> 44.3 (GPT-5-mini CoT vs KARPA)Hurt
add KG retrieval to LLM on Pseudoword splitLLM · HiddenPathQA Pseudowordaccuracy: 23.1 -> 41.7 (GPT-5-mini CoT vs KAPING)Improved
expose all chains at once (Flat) vs depth-by-depth (Tiered) (ablation)Holistic Path Selection · HiddenPathQA (Pseudoword, GPT-5-mini)accuracy: 59.6 (Tiered) vs 51.5 (Flat); Qwen 56.0 vs 56.3Mixed
holistic path selection vs baselines on path retrievalHolistic Path Selection · HiddenPathQAPath F1: 49.2 -> 66.9 (Qwen3-32B, R2-KG vs Flat)Improved

Experimental setup

Benchmarks: HiddenPathQA

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy