Research map / Federated learning and privacy / Privacy attacks and secure learning

FedSum: Data-Efficient Federated Learning Under Data Scarcity Scenario for Text Summarization

AAAI 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Private federated aggregation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FedSum, a federated learning framework for extractive text summarization under data scarcity that partitions samples into prime/normal subsets, skips normal samples during training, and mines knowledge from prototypes and received classification heads.

Problem

Federated extractive summarization with very few training samples (about 2K across the FL system) suffers knowledge shortage and amplified leading position bias, causing performance degradation.

Approach

Extends standard FL along depth and breadth: a collaborative Data Partition separates prime (unbiased, high-loss) from normal samples and a Data skip mechanism drops normal samples with progressively increasing probability; breadth comes from prototype loss on hidden representations and a Portrait Distance Table that weights training with perturbed classification heads from other clients.

Key results

• On CNNDM under Dir=0.1, FedSum reaches R-R 44.98/18.74/36.2 versus 42.8/16.92/34.15 for FedAvg, the strongest gain over FedAvg of about 2.18/1.82/2.05 points in ROUGE recall.
• On PubMed under Dir=0.1, FedSum reaches R-F 31.48/11.2/28.26 and R-R 33.3/11.62/29.86, outperforming FedProx as the best baseline.
• Ablation on CNNDM shows the depth extension alone gives at least 1.01% ROUGE-F gain over FedAvg, and the full two-extension version gives 1.31% to 2.18% gains.

Measured effects reported in the paper

Surprising findings

• Under a uniform (Dir=+inf) data distribution, FedSum is inferior to FedAvg when the data quantity is below 4K samples, per the scalability experiment.
• FedProto underperforms the Separate local-training baseline on CNNDM under heterogeneity.

Limitations

• Evaluated only on extractive summarization; adaptability to other NLP tasks such as machine translation and sentiment analysis is untested.
• Results averaged over only three runs without rigorous statistical analysis.
• Only ROUGE overlap metrics are reported; the paper does not evaluate stability or fairness.

Experimental setup

Datasets: CNN/Daily-Mail WikiHow Reddit PubMed

Benchmarks: CNN/Daily-Mail PubMed

Metrics: ROUGE-1 ROUGE-2 ROUGE-L ROUGE-F1 ROUGE-Recall

Baselines: FedAvg FedProx SCAFFOLD FedNova FedProto FedDC Separate

Builds on: FedAvg FedProto BERTSUM FedDC

Closest papers by meaning

• FedLLM-Bench: Realistic Benchmarks for Federated Learning of Large Language Models (NeurIPS 2024 · similarity 0.84)
• FedWon: Triumphing Multi-domain Federated Learning Without Normalization (ICLR 2024 · similarity 0.84)
• FlowerTune: A Cross-Domain Benchmark for Federated Fine-Tuning of Large Language Models (NeurIPS 2025 · similarity 0.84)
• Beyond Raw Context Transfer: Representation-based Federated Retrieval-Augmented Generation (NeurIPS 2026 · similarity 0.83)
• Splitting with Importance-aware Updating for Heterogeneous Federated Learning with Large Language Models (ICML 2025 · similarity 0.83)
• FedQS: Optimizing Gradient and Model Aggregation for Semi-Asynchronous Federated Learning (NeurIPS 2025 · similarity 0.83)