Research map / Large language models / Retrieval and knowledge
Empty Shelves or Lost Keys? Recall Is the Bottleneck for Parametric Factuality
Nitay Calderon, Eyal Ben-David, Zorik Gekhman, Eran Ofek, Gal Yona
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable thinking | Gemini-3-Pro · WikiProfile (rare facts) | recall gain (points): 20.1 vs 11.3 for popular facts | Improved |
| enable thinking | GPT-5 · WikiProfile (reverse questions) | recall gain (points): 19 vs 12 for direct questions | Improved |
| enable thinking | frontier LLMs · WikiProfile (popularity gap) | recall gap (points): 21.4 -> 12.5 | Improved |
| enable thinking | frontier LLMs · WikiProfile (directionality gap) | recall gap (points): 9 -> 2 | Improved |
| enable thinking on encoded vs non-encoded facts | thinking-optimized LLMs · WikiProfile | facts recovered (%): 40-65 vs 5-15 | Improved |
| scale Gemma3 from 1B to 27B | Gemma3 · WikiProfile | encoding failures (%): 85 -> 23 | Improved |
| scale Gemma3 model size | Gemma3 · WikiProfile | recall failure share (%): peaking at 33 (with thinking) and 40 (without) | Hurt |
| switch from closed-book generation to multiple-choice verification for reverse questions | 13 LLMs · WikiProfile | reverse vs direct recall: reverse easier for 9 models, similar for 4 | Improved |
| use reverse instead of direct questions (closed-book) | GPT-5 · WikiProfile | direct recall (%): 82.9 -> 74 | Hurt |
| use rare instead of popular facts | Gemini-3-Pro · WikiProfile | direct recall among encoded (%): 84.6 -> 63.3 | Hurt |
| use rare instead of popular facts | Gemini-3-Pro · WikiProfile | encoding rate (%): 99.5 -> 94.5 | Hurt |
| high-verbatim vs natural phrasing | 13 LLMs · WikiProfile | performance distribution: no significant effects in 104 tests | No clear effect |
Closest papers by meaning
- The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations (NeurIPS 2026 · similarity 0.88)
- Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness (NeurIPS 2025 · similarity 0.88)
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.87)
- OntoFact: Unveiling Fantastic Fact-Skeleton of LLMs via Ontology-Driven Reinforcement Learning (AAAI 2024 · similarity 0.87)
- Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation (NeurIPS 2025 · similarity 0.87)
- ListQA: A Benchmark for Evaluating List-Formatted Factual Knowledge Retrieval in Large Language Models (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
