Research map / Large language models / Language, safety and interpretability
PaLM: Scaling Language Modeling with Pathways
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi and 55 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add chain-of-thought prompting (ablation) | PaLM 540B · GSM8K 8-shot | accuracy: 17% -> 54% | Improved |
| add an external calculator (ablation) | PaLM 540B + chain-of-thought · GSM8K 8-shot | accuracy: 54% -> 58% | Improved |
| scale PaLM from 62B to 540B | PaLM + chain-of-thought · GSM8K 8-shot | accuracy: 33% -> 54% | Improved |
| scale PaLM from 8B to 540B | PaLM · MMLU 5-shot | average accuracy: 25.3 -> 69.3 | Improved |
| scale PaLM from 8B to 540B | PaLM · 29 NLP benchmarks 1-shot | avg NLG: 41.5 -> 63.9 | Improved |
| scale PaLM from 62B to 540B | PaLM · 29 NLP benchmarks 1-shot | avg NLU: 67.3 -> 74.7 | Improved |
| finetune on SuperGLUE instead of few-shot | PaLM 540B · SuperGLUE dev | CB accuracy / WiC: WiC 64.6 -> 78.8 | Improved |
| finetune on SuperGLUE instead of few-shot | PaLM 540B · SuperGLUE dev | BoolQ: 89.1 -> 92.2 | Improved |
| use PaLM instead of Chinchilla | MMLU 5-shot · MMLU Other category | accuracy: 73.9 -> 69.6 | Hurt |
| use PaLM 540B instead of Chinchilla 70B | MMLU 5-shot · MMLU average | accuracy: 67.5 -> 69.3 | Improved |
| use PaLM few-shot instead of prior SOTA 0-shot | PaLM 540B · Natural Questions 0-shot | EM: 24.7 -> 21.2 | Hurt |
| use PaLM 540B with chain-of-thought instead of GPT-3 finetuned with verifier | GSM8K · GSM8K 8-shot | accuracy: 55% -> 58% | Improved |
Later papers that compared with this method
1 later accepted paper reports PaLM 540B as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.
- On the Power of Decision Trees in Auto-Regressive Language Modeling (NeurIPS 2024 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Scaling Inference-Efficient Language Models (ICML 2025 · similarity 0.89)
- A Hitchhiker's Guide to Scaling Law Estimation (ICML 2025 · similarity 0.89)
- Cross-model Control: Improving Multiple Large Language Models in One-time Training (NeurIPS 2024 · similarity 0.88)
- Rethinking Optimization and Architecture for Tiny Language Models (ICML 2024 · similarity 0.88)
- Pruning Large Language Models with Semi-Structural Adaptive Sparse Training (AAAI 2025 · similarity 0.88)
- Sloth: scaling laws for LLM skills to predict multi-benchmark performance across families (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
