Research map / Large language models / Retrieval and knowledge
SynthWorlds: Controlled Parallel Worlds for Disentangling Reasoning and Knowledge in Language Models
Ken Gu, Advait Bhat, Mike A Merrill, Robert West, Xin Liu, Daniel McDuff, Tim Althoff
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add one-step RAG | LMs on SynthWorlds multi-hop QA · SynthWorlds multi-hop QA | knowledge advantage gap: -4.0 (GPT-5-mini), -1.3 (Gemini-2.0-Flash) | Hurt |
| use IRCoT + RAG instead of one-step RAG | LMs on SynthWorlds multi-hop QA · SynthWorlds multi-hop QA | knowledge advantage gap: 5.2 (GPT-5-mini), 10.3 (Gemini-2.0-Flash) | Improved |
| use IRCoT + RAG instead of one-step RAG | gpt-oss models · SynthWorlds multi-hop QA | F1 | No clear effect |
| give agent page content in addition to links | LM agents · SynthWorlds page navigation | knowledge advantage gap: 9.3 (GPT-5-mini), 7.0 (Gemini-2.0-Flash) | Improved |
| give agent page content in addition to links | LM agents · SynthWorld-RM page navigation | external entity mention rate: 48% -> 35% (GPT-5-mini); 60% -> 15% (Gemini-2.0-Flash) | Improved |
| provide gold documents (reading comprehension) instead of closed-book | LMs · SynthWorlds multi-hop QA | F1 | Improved |
Closest papers by meaning
- MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models (AAAI 2026 · similarity 0.89)
- Finding the Minimal Parameter Budget for Implicit Reasoning: A Data Complexity Driven Scaling Law for Language Models (ICML 2026 · similarity 0.89)
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.89)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.89)
- Improving Factuality and Reasoning in Language Models through Multiagent Debate (ICML 2024 · similarity 0.88)
- Calibrating Reasoning in Language Models with Internal Consistency (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
