Research map / Large language models / Retrieval and knowledge

Autofocus Retrieval: An Effective Pipeline for Multi-Hop Question Answering With Semi-Structured Knowledge

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: Explanatory and multiple-choice QA benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AF-Retriever, a modular zero-shot pipeline combining LLM-based Cypher-style constraint parsing, incremental scope expansion over candidate entities, hybrid graph and vector retrieval, and LLM reranking for multi-hop QA over semi-structured knowledge bases.

Problem

Multi-hop question answering over semi-structured knowledge bases (SKBs) that link knowledge graph nodes to unstructured text, evaluated on the STaRK benchmarks, where existing methods either rely on task-specific fine-tuning or focus on single retrieval techniques.

Approach

Combines LLM-derived target type and Cypher-like triplets with regex parsing, incremental expansion of constant candidates until k answers satisfy relational constraints, a hybrid merge with a VSS branch, and final LLM reranking (pointwise, pairwise, listwise compared), all without task-specific training.

Key results

• AF-Retriever reaches hit@1 of 46.2 on PRIME, 78.6 on MAG and 61.2 on AMAZON (synthetic test sets), the best zero- and one-shot results on all three STaRK datasets.
• Averaged over the synthetic sets, hit@1 of 62.0 vs 46.9 for the second-best 4StepFocus (per the paper's table), with the abstract claiming a 32.1% average first-hit improvement over the second-best method.
• On PRIME the supervised GraphRAFT achieves 39.7% higher hit@1 than AF-Retriever, the one case where a trained method wins.

Surprising findings

• Pairwise LLM reranking generally beats listwise and pointwise, but listwise is best on MAG with GPT-5 mini and GPT OSS 120B.
• Including relational information in node embeddings helps direct retrieval but hurts retrieval of non-target candidates on PRIME and MAG.

Limitations

• Relies on LLM Cypher generation; 4% of queries had no valid triplet and 8% had no constant, making graph search unsuitable.
• Pairwise reranking is expensive, with runtimes up to several minutes in some cases.
• Performance on PRIME lags supervised GraphRAFT, suggesting out-of-the-box LLMs struggle with specialized medical entities.

Experimental setup

Datasets: STaRK-PRIME STaRK-MAG STaRK-AMAZON

Benchmarks: STaRK-PRIME STaRK-MAG STaRK-AMAZON

Metrics: hit@1 hit@5 hit@20 mrr recall@20

Baselines: BM25 VSS (Ada-002) VSS (Multi-Ada) DPR VSS + Reranker QA-GNN ToG AvaTaR 4StepFocus KAR HybGRAG ReAct

Builds on: STaRK VSS+Reranker 4StepFocus ReAct Reflexion

Closest papers by meaning

• Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.89)
• Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation (AAAI 2025 · similarity 0.89)
• Question Calibration and Multi-Hop Modeling for Temporal Question Answering (AAAI 2024 · similarity 0.87)
• Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.87)
• Tree-of-Reasoning Question Decomposition for Complex Question Answering with Large Language Models (AAAI 2024 · similarity 0.87)
• Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025 · similarity 0.87)