AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

PaLM: Scaling Language Modeling with Pathways

Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi and 55 more

JMLR 2023 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 5,432 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add chain-of-thought prompting (ablation)PaLM 540B · GSM8K 8-shotaccuracy: 17% -> 54%Improved
add an external calculator (ablation)PaLM 540B + chain-of-thought · GSM8K 8-shotaccuracy: 54% -> 58%Improved
scale PaLM from 62B to 540BPaLM + chain-of-thought · GSM8K 8-shotaccuracy: 33% -> 54%Improved
scale PaLM from 8B to 540BPaLM · MMLU 5-shotaverage accuracy: 25.3 -> 69.3Improved
scale PaLM from 8B to 540BPaLM · 29 NLP benchmarks 1-shotavg NLG: 41.5 -> 63.9Improved
scale PaLM from 62B to 540BPaLM · 29 NLP benchmarks 1-shotavg NLU: 67.3 -> 74.7Improved
finetune on SuperGLUE instead of few-shotPaLM 540B · SuperGLUE devCB accuracy / WiC: WiC 64.6 -> 78.8Improved
finetune on SuperGLUE instead of few-shotPaLM 540B · SuperGLUE devBoolQ: 89.1 -> 92.2Improved
use PaLM instead of ChinchillaMMLU 5-shot · MMLU Other categoryaccuracy: 73.9 -> 69.6Hurt
use PaLM 540B instead of Chinchilla 70BMMLU 5-shot · MMLU averageaccuracy: 67.5 -> 69.3Improved
use PaLM few-shot instead of prior SOTA 0-shotPaLM 540B · Natural Questions 0-shotEM: 24.7 -> 21.2Hurt
use PaLM 540B with chain-of-thought instead of GPT-3 finetuned with verifierGSM8K · GSM8K 8-shotaccuracy: 55% -> 58%Improved

Later papers that compared with this method

1 later accepted paper reports PaLM 540B as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy