Research map / Large language models / Retrieval and knowledge
Domain-specific QA datasets
Build question answering and text-to-SQL resources for medical and scientific domains.
79 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Retrieval and knowledge, keyword cluster “question · answering · editing”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
18 papers in 2023–24 and 35 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 7.9% to 5.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · ClaimEval: Integrated and Flexible Framework for Claim Evaluation Using Credibility of Sources (AAAI 2016 · 35 citations)
- 2017 · Open-Vocabulary Semantic Parsing with both Distributional Statistics and Formal Knowledge (AAAI 2017 · 10 citations)
- 2018 · OTyper: A Neural Architecture for Open Named Entity Typing (AAAI 2018 · 26 citations)
- 2019 · A Pattern-Based Approach to Recognizing Time Expressions (AAAI 2019)
- 2020 · Iteratively Questioning and Answering for Interpretable Legal Judgment Prediction (AAAI 2020 · 108 citations)
- 2021 · Judgment Prediction via Injecting Legal Knowledge into Neural Networks (AAAI 2021 · 58 citations)
- 2022 · OneRel: Joint Entity and Relation Extraction with One Module in One Step (AAAI 2022 · 187 citations)
- 2023 · Factual and Informative Review Generation for Explainable Recommendation (AAAI 2023 · 17 citations)
- 2024 · A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules? (ICML 2024 · 10 citations)
- 2025 · HLMEA: Unsupervised Entity Alignment Based on Hybrid Language Models (AAAI 2025 · 4 citations)
- 2026 · WebShaper: Agentically Data Synthesizing via Information-Seeking Formalization (ICLR 2026 · 2 citations)
Earliest papers here
- ClaimEval: Integrated and Flexible Framework for Claim Evaluation Using Credibility of Sources (AAAI 2016 · 35 citations)
- Open-Vocabulary Semantic Parsing with both Distributional Statistics and Formal Knowledge (AAAI 2017 · 10 citations)
- Trust-Sensitive Evolution of DL-Lite Knowledge Bases (AAAI 2017 · 5 citations)
What has been tried
52 papers here report 510 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 7%, hurt 18%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| use the just-smiles prompt instead of other prompts | molecular BO tasks · BO performance | Helped | A Sober Look at LLMs for Material Discovery: Are They Actually Good for Bayesian Optimization Over Molecules? (ICML 2024 · 10 citations) | |
| add the label refiner | EN-FR-15K · MRR | 61.9±0.8 -> 81.0±0.3 | Helped | Entity Alignment with Noisy Annotations from Large Language Models (NeurIPS 2024 · 10 citations) |
| Use PF-IKF rather than random triple selection for TRE generation | DBP15KZH−EN · Hit@1 | Helped | HLMEA: Unsupervised Entity Alignment Based on Hybrid Language Models (AAAI 2025 · 4 citations) | |
| Replace UoT's uncertainty-based reward with heuristic self-evaluation | 20 Questions, Medical Diagnosis, and Troubleshooting · success rate | 54.4 -> 48.5; 91.7 -> 89.0; 67.3 -> 66.0 | Hurt | Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in LLMs (NeurIPS 2024 · 4 citations) |
| Select 5 few-shot examples per question using BM25 similarity instead of fixed examples | SQL (PostgreSQL) · execution accuracy | +32.25 | Helped | SM3-Text-to-Query: Synthetic Multi-Model Medical Text-to-Query Benchmark (NeurIPS 2024 · 4 citations) |
| add external gene embeddings to sciLaMA instead of using self-informed embeddings | Human pancreatic datasets · ARI | 0.436 -> 0.522 | Helped | sciLaMA: A Single-Cell Representation Learning Framework to Leverage Prior Knowledge from Large Language Models (ICML 2025 · 3 citations) |
| Replace stratified sampling with uniform random sampling | Aggregation queries over three text datasets · estimation variance | Hurt | UQE: A Query Engine for Unstructured Databases (NeurIPS 2024 · 3 citations) | |
| Add personalized visit history context to the LLM prompt | MIMIC-III · AUPRC | 0.2893 -> 0.3079 | Helped | DearLLM: Enhancing Personalized Healthcare via Large Language Models-Deduced Feature Correlations (AAAI 2025 · 2 citations) |
| replace CQG question generation with QAEmb example-based question generation | STS Avg. · Spearman correlation | 77.60 vs. 65.20 | Hurt | A General Framework for Producing Interpretable Semantic Text Embeddings (ICLR 2025 · 2 citations) |
| add encoder fine-tuning and natural-language trajectory summaries | CF · MSE | 76.3 ± 1.61 -> 55.3 ± 0.811 | Helped | Continuously Updating Digital Twins using Large Language Models (ICML 2025 · 1 citation) |
| Provide gold time filtering and table identification, in addition to gold entities | MIMIC-III, 25% test set · recall | 76.11% -> 82.49% | Helped | EHRCon: Dataset for Checking Consistency between Unstructured Notes and Structured Tables in Electronic Health Records (NeurIPS 2024 · 1 citation) |
| Iterate question generation and use earlier Insight Cards as few-shot examples | Insight Card generation · number of unique Insight Cards | Helped | Question-guided Insights Generation for Automated Exploratory Data Analysis (AAAI 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 22 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MIMIC-III · 2 of 22 papers
- PubChem · 2 of 22 papers
- MIMIC-IV discharge summaries · 2 of 22 papers
Baselines
- GPT-4o · 2 of 22 papers
- BERT · 2 of 22 papers
- BM25 · 2 of 22 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 53 papers; from OpenAlex and the papers' affiliations).
- Korea Advanced Institute of Science and Technology · 5 papers
- Microsoft · 3 papers
- University of Washington · 3 papers
- University of Cambridge · 2 papers
- IBM · 2 papers
- The Hong Kong Polytechnic University · 2 papers
- Beijing University of Posts and Telecommunications · 2 papers
- University College London · 2 papers
- Sungkyunkwan University · 2 papers
- Technology · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Knowledge graph reasoning with LLMs (174 papers)
- Knowledge editing in LLMs (170 papers)
- Knowledge-augmented question answering (134 papers)
- Explanatory and multiple-choice QA benchmarks (152 papers)
Elsewhere in the topic
- RAG robustness and evaluation (164 papers)
- Retrieval-augmented reasoning (130 papers)
- Long-context and personalized retrieval (61 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).