Research map / Large language models / Agents, code and math
Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning
Maxwell Nye, Michael Henry Tessler, Joshua B. Tenenbaum, Brenden M. Lake
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add System 2 world model parsing of GPT-3 output | GPT-3 · bAbI Task 2 QA | accuracy: 29.0 -> 100 | Improved |
| replace world model with off-the-shelf neural NLI scoring (ablation) | GPT-3 · bAbI Task 2 QA | accuracy: 29.0 -> 32.5 | No clear effect |
| add world model rejection of inconsistent sentences | GPT-3 story generation · bAbI stories | sentences rejected: 53.1% of sentences rejected | Improved |
| add dual-system world model resampling | BART · CLUTRR, prompt from model | % stories without error: 36.4 -> 93.5 | Improved |
| add dual-system world model resampling | BART · CLUTRR, prompt from dataset | % stories without error: 60 -> 96.1 | Improved |
| add dual-system world model resampling | BART · CLUTRR, prompt from model | % lines without error: 71.9 -> 96.3 | Improved |
| use dual-system with world model instead of single-system | BART · CLUTRR human judgment | proportion preferring dual-system: 0.84 and 0.79 | Improved |
| use neural NLI as System 2 instead of world model (ablation) | BART · CLUTRR human judgment | proportion preferring dual-system: 0.51 and 0.58 | No clear effect |
| reduce sampling budget from 10 to 5 (ablation) | dual-system world model · CLUTRR, prompt from model | % stories without error: 93.5 -> 88.3 | Hurt |
| reduce sampling budget from 10 to 5 (ablation) | dual-system world model · CLUTRR, prompt from dataset | % stories without error: 96.1 -> 93.5 | Hurt |
| reduce sampling budget from 10 to 5 (ablation) | dual-system world model · CLUTRR, prompt from model | % lines without error: 96.3 -> 94.6 | Hurt |
| add target-location consistency check with execution model | LSTM · gSCAN random, 5000 examples | exact match: 57.2 -> 74.7 | Improved |
Closest papers by meaning
- Logically Consistent Language Models via Neuro-Symbolic Integration (ICLR 2025 · similarity 0.88)
- Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models (ICML 2025 · similarity 0.87)
- Right for the Right Reasons: Avoiding Reasoning Shortcuts via Prototypical Neurosymbolic AI (NeurIPS 2025 · similarity 0.87)
- Deciphering Raw Data in Neuro-Symbolic Learning with Provable Guarantees (AAAI 2024 · similarity 0.87)
- NeSTR: A Neuro-Symbolic Abductive Framework for Temporal Reasoning in Large Language Models (AAAI 2026 · similarity 0.87)
- Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
