Research map / Label-efficient and robust learning / Self-supervised and contrastive
PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PEGASUS-large pre-trained on HugeNews rather than the previous reported system | Summarization models · XSum | ROUGE-1/2/L F1: 45.14/22.27/37.25 -> 47.21/24.56/39.25 | Helped |
| Select important principal sentences rather than random gaps (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.28/16.23/31.21 -> 39.79/16.58/31.70 | Helped |
| Select important principal sentences rather than lead sentences (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.22/16.12/31.09 -> 39.79/16.58/31.70 | Helped |
| Use MLM alone rather than GSG Ind-Orig (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.79/16.58/31.70 -> 37.22/14.48/29.62 | Hurt |
| Add MLM to GSG Ind-Orig (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.79/16.58/31.70 -> 39.08/16.21/31.20 | Hurt |
| Add MLM to GSG Ind-Orig at early checkpoints (ablation) | PEGASUS-base · Four downstream datasets | ROUGE | Helped |
| Add MLM to GSG Ind-Orig at 500k pre-training steps (ablation) | PEGASUS-base · Four downstream datasets | ROUGE | Hurt |
| Increase gap-sentences ratio to 75% (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.61/16.51/31.48 -> 36.94/14.21/29.14 | Hurt |
| Use 15% rather than 30% gap sentences (ablation) | PEGASUS-base · CNN/DailyMail | ROUGE-1/2/L F1: 41.83/18.82/38.96 -> 41.88/18.98/38.97 | Helped |
| Use 30% rather than 15% gap sentences (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.47/16.32/31.30 -> 39.61/16.51/31.48 | Helped |
| Pre-train on HugeNews rather than C4 (ablation) | PEGASUS-base · XSum | ROUGE-1/2/L F1: 39.79/16.58/31.70 -> 41.63/18.47/33.48 | Helped |
| Pre-train on HugeNews rather than C4 (ablation) | PEGASUS-base · WikiHow | ROUGE-1/2/L F1: 36.58/15.64/30.01 -> 34.93/14.67/28.63 | Hurt |
Closest papers by meaning
- Towards Improving Faithfulness in Abstractive Summarization (NeurIPS 2022 · similarity 0.87)
- Meta-Transfer Learning for Low-Resource Abstractive Summarization (AAAI 2021 · similarity 0.87)
- Distilling Structured Rationale from Large Language Models to Small Language Models for Abstractive Summarization (AAAI 2025 · similarity 0.86)
- Improving Faithfulness in Abstractive Text Summarization with EDUs Using BART (Student Abstract) (AAAI 2024 · similarity 0.85)
- PersonalSum: A User-Subjective Guided Personalized Summarization Dataset for Large Language Models (NeurIPS 2024 · similarity 0.85)
- Automatic Summarization of Long Documents (Student Abstract) (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.