Research map / Label-efficient and robust learning / Recognition and long-tail

Unsupervised Extractive Summarization with Learnable Length Control Strategies

AAAI 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Multilingual language representation pretraining · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an unsupervised extractive summarizer built on a SimSiam-style siamese network with bidirectional prediction objectives (Rest-Ext, Ext-Doc) and a differentiable knapsack transformer that approximates the DP solver for length-controllable extraction.

Problem

Unsupervised extractive summarization relies on non-differentiable centrality-based ranking (TextRank, PACSUM) that needs positional priors, and length-controllable extractive summarization lacks end-to-end trainable solutions for the knapsack-style selection under length budgets.

Approach

Replace centrality scoring with a bidirectional prediction criterion in a siamese network (rest-to-selected and selected-to-document prediction), make top-K/knapsack selection differentiable via Gumbel-softmax and straight-through estimators, and train a BERT-like Knapsack Transformer on simulated data to approximate the dynamic programming solver so length control can be learned end-to-end.

Key results

• On CNNDM, the unsupervised model with both objectives reaches 40.92 ROUGE-1 and 37.27 ROUGE-L, comparable to FAR and PACSUM and well above TextRank (SimCSE) at 33.36 ROUGE-1.
• The Knapsack Transformer (DP-trained) achieves 0.7% per-item error and 93.8% exact match to DP on CNewSum.
• Under length control on CNNDM at 500 characters, the joint-trained Ours+KTDP+DP gets 40.5 ROUGE-1 versus 32.5 for TextRank (tf-idf)+DP.

Measured effects reported in the paper

Surprising findings

• Rest-Ext alone already performs close to the full model on CNNDM and NYT, while Ext-Doc alone is less stable in training.
• In human evaluation, the unsupervised model with knapsack transformer scores higher than the supervised BertSum on relevance and consistency despite lower ROUGE in some settings.

Limitations

• Length control is evaluated only with DP at inference, and the knapsack module approximates DP imperfectly (82-94% exact match on CNNDM/CNewSum).
• Evaluated only on news datasets (CNNDM, NYT, CNewSum), with no other domains tested.

Experimental setup

Datasets: CNN/DailyMail NYT CNewSum

Benchmarks: CNN/DailyMail NYT CNewSum

Metrics: ROUGE-1 ROUGE-2 ROUGE-L human rating (relevance, coherence, consistency)

Baselines: TextRank (tf-idf) TextRank (BERT) TextRank (SimCSE) PACSUM FAR STAS TED Lead-3 BertSum

Builds on: SimSiam TextRank PACSUM BertSum

Closest papers by meaning

• SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents (AAAI 2017 · similarity 0.88)
• Flexible Non-Autoregressive Extractive Summarization with Threshold: How to Extract a Non-Fixed Number of Summary Sentences (AAAI 2021 · similarity 0.88)
• Text Summarization with Oracle Expectation (ICLR 2023 · similarity 0.87)
• Unity in Diversity: Learning Distributed Heterogeneous Sentence Representation for Extractive Summarization (AAAI 2018 · similarity 0.87)
• BooookScore: A systematic exploration of book-length summarization in the era of LLMs (ICLR 2024 · similarity 0.84)
• Document Summarization with Conformal Importance Guarantees (NeurIPS 2025 · similarity 0.83)