Research map / Label-efficient and robust learning / Recognition and long-tail
Sequence Level Contrastive Learning for Text Summarization
Shusheng Xu, Xingxing Zhang, Yi Wu, Furu Wei
This paper tried a common change: Add a contrastive loss
Add contrastive learning · CNNDM and NYT human evaluation · Mean faithfulness rank: 2.62 -> 2.42; 2.68 -> 2.39 Improved
What 1,317 accepted papers report for this change
- 1,253 improved
- 22 no clear effect
- 16 mixed
- 26 hurt
- Beyond Looking Up, Try Looking Around: Harmonizing Global Structure and Local Consistency in Optimal Transport for Short Text Clustering ICML 2026 · add contrastive learning · SearchSnippets · ACC: 56.65 -> 80.12Improved
- SleepLM: Natural-Language Intelligence for Human Sleep ICML 2026 · add contrastive objective to captioning (CoCa vs Cap) · sleep event IoU · IoU: 29.2 -> 27.4Hurt
- LIMSSR: LLM-Driven Sequence-to-Score Reasoning under Training-Time Incomplete Multimodal Observations ICML 2026 · add contrastive loss L_con · FS1000 average · Spearman correlation: 0.759 -> 0.789Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add sequence-level contrast between document and gold summary | BART Large on XSum · XSum test | ROUGE-1: 45.35 -> 45.65 | Improved |
| Add sequence-level contrast between document and gold summary | BART Large on XSum · XSum test | ROUGE-2: 22.01 -> 22.41 | Improved |
| Add sequence-level contrast between document and gold summary | BART Large on XSum · XSum test | ROUGE-L: 36.76 -> 37.04 | Improved |
| Add contrast between document and generated summary | BART Large on NYT · NYT test | ROUGE-1: 53.20 -> 54.25 | Improved |
| Add contrast between document and generated summary | BART Large on NYT · NYT test | ROUGE-2: 35.04 -> 35.82 | Improved |
| Add contrast between document and generated summary | BART Large on NYT · NYT test | ROUGE-L: 49.23 -> 50.24 | Improved |
| Use generated summaries rather than gold summaries as contrast partners | SeqCo on CNNDM and NYT · CNNDM and NYT test | ROUGE | Improved |
| Use generated summaries rather than gold summaries as contrast partners | SeqCo on XSum · XSum test | ROUGE | Hurt |
| Replace multi-head attention similarity with [CLS] pooling (ablation) | SeqCo on CNNDM · CNNDM validation | ROUGE-1: 45.88 -> 45.72 | Hurt |
| Add multiple contrastive pairs instead of one (ablation) | SeqCo on CNNDM · CNNDM validation | ROUGE-1: 45.88 -> 45.72 | Hurt |
| Use decoder-conditioned representations for contrastive learning (ablation) | SeqCo on CNNDM · CNNDM validation | ROUGE-L: 42.66 -> 41.55 | Hurt |
| Add multiple contrastive pairs instead of one (ablation) | SeqCo on XSum · XSum validation | ROUGE-2: 22.42 -> 22.52 | Improved |
Closest papers by meaning
- Sequence-to-Sequence Contrastive Learning for Text Recognition (CVPR 2021 · similarity 0.87)
- SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents (AAAI 2017 · similarity 0.86)
- Towards Improving Faithfulness in Abstractive Summarization (NeurIPS 2022 · similarity 0.85)
- Boosting Semi-Supervised Scene Text Recognition via Viewing and Summarizing (NeurIPS 2024 · similarity 0.85)
- Improving Faithfulness in Abstractive Text Summarization with EDUs Using BART (Student Abstract) (AAAI 2024 · similarity 0.84)
- Retrieval-Enhanced Contrastive Vision-Text Models (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
