Research map / Large language models / AI and society
Think Big, Teach Small: Do Language Models Distil Occam’s Razor?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add 2 additional examples (AS I) to optimal witness set | GPT-3D-T0 · 300 concepts | mean accuracy (%): 18.44 -> 30.87 | Helped |
| Add 3 more examples (AS II) after AS I | GPT-3D-T0 · 300 concepts | mean accuracy (%): 30.87 -> 43.40 | Helped |
| Increase concept complexity from VL-C to VH-C | GPT-3D-T0 · 300 concepts | mean accuracy (%): 53.00 -> 17.18 | Hurt |
| Very low complexity concepts, AS II vs WS | GPT-3D-T0 · 300 concepts VL-C | accuracy (%): 37.00 -> 75.00 | Helped |
| Very high complexity concepts, AS II vs WS | GPT-3D-T0 · 300 concepts VH-C | accuracy (%): 10.02 -> 22.35 | Helped |
| Temperature 0 instead of 1 (ablation) | GPT-3 (8 configurations) · 8 concepts | accuracy | Helped |
| Larger model size (Ada to Da Vinci) (ablation) | GPT-3 · 8 concepts | accuracy | No clear effect |
| Use Da Vinci model on easy concepts with few examples (ablation) | GPT-3D · 8 concepts, WS and VL-C | accuracy | Helped |
| Ensemble (majority) of 40 samples instead of averaging (ablation) | GPT-2 · 8 concepts | accuracy | Helped |
| Concepts without loops instead of with loops | GPT-3 · supplementary concept sets | accuracy | Helped |
| Prompt GPT-3 to explain learnt concepts | GPT-3 · 8 concepts | explanation quality | No clear effect |
| Increase concept complexity in human explanations | Human learners · 8 concepts | fraction of correct explanations | Hurt |
Closest papers by meaning
- In-Context Learning and Occam's Razor (ICML 2025 · similarity 0.88)
- Quantifying the Gain in Weak-to-Strong Generalization (NeurIPS 2024 · similarity 0.87)
- Algorithmic progress in language models (NeurIPS 2024 · similarity 0.87)
- Training on the Test Task Confounds Evaluation and Emergence (ICLR 2025 · similarity 0.87)
- From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generation (ICLR 2025 · similarity 0.87)
- Does Reasoning Emerge? Examining the Probabilities of Causation in Large Language Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.