Research map / Large language models / Retrieval and knowledge

TrustUQA: A Trustful Framework for Unified Structured Data Question Answering

AAAI 2025 · Large language models › Retrieval and knowledge · approach: Explanatory and multiple-choice QA benchmarks · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TrustUQA, a trustful NL2Query framework that unifies table, KG and temporal KG QA by translating data into a Condition Graph and querying it via LLM-written function calls, few-shot prompting, and dynamic demonstration retrieval.

Problem

Natural language QA over multiple types of structured data (tables, KGs, temporal KGs) is hard to unify: existing NL2Query methods are tied to one query language, while RAG-based unified methods are less trustworthy because answers rest on retrieved evidence exposed to the LLM.

Approach

Represent tables, KGs and temporal KGs in a unified Condition Graph of condition triples, have the LLM write simple get_information-style LLM queries with few-shot demonstrations, and translate them by fixed rules into executable queries; demonstrations are chosen dynamically by retrieving similar training questions whose executed queries match the gold answer.

Key results

• TrustUQA reaches 85.9% denotation accuracy on WikiSQL, beating MAPO and TAPAS and the unified RAG baselines StructGPT and Readi, but trails Readi on WTQ (44.2% vs 66.7%).
• On WebQSP, TrustUQA achieves 83.5% Hit@1, the best among the compared methods, versus 75.1% for UniKGQA and 74.3% for Readi.
• On CronQuestions, TrustUQA reaches 97.2% Hits@1 overall, state of the art among the compared methods, slightly above LGQA (96.9%).

Measured effects reported in the paper

Surprising findings

• On WTQ, TrustUQA (44.2%) underperforms the unified baseline Readi (66.7%) and several table-specific models.
• Removing dynamic demonstration retrieval lowers WebQSP accuracy from 83.5% to 75.1%, while MetaQA slightly improves without it (98.1 vs 97.8).

Limitations

• Weak on WTQ due to multi-answer questions, non-standard table formats, and entity mapping ambiguity.
• Errors are dominated by query generation mistakes and relation/entity mapping errors.
• Mixed-source and across-source results depend on correct demonstration retrieval and drop noticeably relative to single-source settings.

Experimental setup

Datasets: WikiSQL WTQ WebQSP MetaQA CronQuestions

Benchmarks: WikiSQL WTQ WebQSP MetaQA CronQuestions

Metrics: denotation accuracy Hit@1 set comparison accuracy accuracy on mixed and across-source QA

Baselines: MAPO TAPAS UnifiedSKG TAPEX DATER StructGPT Readi KV-Mem GraftNet EmbedKGQA NSM UniKGQA

Builds on: StructGPT Readi SentenceBERT

Closest papers by meaning

• LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake (ICML 2026 · similarity 0.89)
• FlexKBQA: A Flexible LLM-Powered Framework for Few-Shot Knowledge Base Question Answering (AAAI 2024 · similarity 0.88)
• SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs (ICLR 2024 · similarity 0.87)
• Diagnosing and Addressing Pitfalls in KG-RAG Datasets: Toward More Reliable Benchmarking (NeurIPS 2025 · similarity 0.87)
• KBQA-o1: Agentic Knowledge Base Question Answering with Monte Carlo Tree Search (ICML 2025 · similarity 0.87)
• Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.86)