Research map / Large language models / Language, safety and interpretability
Measuring abstract reasoning in neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a relation-network architecture to compare context panels and candidate answers | PGM neutral split; WReN versus standard baselines · PGM neutral test | accuracy: 42.0 -> 62.6 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM neutral test | accuracy: 62.6 -> 76.9 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM interpolation test | accuracy: 64.4 -> 67.4 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM held-out attribute pairs test | accuracy: 27.2 -> 51.7 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM held-out triple pairs test | accuracy: 41.9 -> 56.3 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM held-out triples test | accuracy: 19.0 -> 20.1 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM held-out line-type test | accuracy: 14.4 -> 16.4 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM held-out shape-colour test | accuracy: 12.5 -> 13.0 | Helped |
| Add auxiliary symbolic meta-target training (ablation) | WReN · PGM extrapolation test | accuracy: 17.2 -> 15.5 | Hurt |
| Remove distracting attribute values (ablation) | WReN · PGM test | accuracy: 62.6% -> 78.3% | Helped |
Closest papers by meaning
- Emergent Symbolic Mechanisms Support Abstract Reasoning in Large Language Models (ICML 2025 · similarity 0.88)
- Boule or Baguette? A Study on Task Topology, Length Generalization, and the Benefit of Reasoning Trace (NeurIPS 2026 · similarity 0.88)
- Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance (NeurIPS 2024 · similarity 0.87)
- Attention as a Hypernetwork (ICLR 2025 · similarity 0.87)
- Phenomenal Yet Puzzling: Testing Inductive Reasoning Capabilities of Language Models with Hypothesis Refinement (ICLR 2024 · similarity 0.87)
- On the Inductive Bias of Stacking Towards Improving Reasoning (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.