Research map / Label-efficient and robust learning / Recognition and long-tail
Compositional Generalization for Multi-Label Text Classification: A Data-Augmentation Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Evaluates compositional generalization of multi-label text classifiers on new label-composition splits of SemEval, AAPD and IMDB, and proposes LS-PT and LD-VAE generators for compositional data augmentation.
Problem
Existing multi-label text classifiers fail to generalize to rare or unseen compositions of individual labels, since standard random splits let all label compositions appear in training.
Approach
Build compositional splits where test label compositions are unseen in training, and augment training with synthetic texts for novel label compositions produced by two disentangled conditional generators (label-specific prefix-tuning and a label-disentangled VAE), filtered by a BERT-based quality control classifier.
Key results
- On AAPD compositional split, BERT accuracy drops from 37.79 (iid) to 22.02 (CG), and T5+CLP drops to 9.17 on CG.
- LD-VAE augmentation raises BERT accuracy on AAPD CG split from 21.96 to 31.11 and Jaccard from 52.24 to 58.50.
- LS-PT and LD-VAE outperform Flan-T5, VAE-DPrior, GPT-3.5 and GPT2-PT as augmentation generators across SemEval, AAPD and IMDB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LD-VAE-generated synthetic data for classifier training | BERT on AAPD CG · AAPD CG | accuracy: 21.96 -> 31.11 | Helped |
| Use LD-VAE-generated synthetic data for classifier training | BERT on SemEval CG · SemEval CG | accuracy: +5.52 | Helped |
| Use LD-VAE-generated synthetic data for classifier training | BERT on IMDB CG · IMDB CG | accuracy: +7.27 | Helped |
| Use LS-PT rather than the strongest listed baseline generator | BERT on SemEval CG · SemEval CG | accuracy: 8.01 -> 8.29 | Helped |
| Use LD-VAE rather than the strongest listed baseline generator | BERT on AAPD CG · AAPD CG | accuracy: 28.03 -> 31.11 | Helped |
| Use LS-PT rather than the strongest listed baseline generator | BERT on IMDB CG · IMDB CG | accuracy: 11.69 -> 12.56 | Helped |
| Select synthetic examples with the filter instead of randomly (ablation) | LS-PT augmentation · SemEval CG | accuracy: 6.93 -> 8.29 | Helped |
| Select synthetic examples with the filter instead of randomly (ablation) | LD-VAE augmentation · SemEval CG | accuracy: 6.43 -> 8.44 | Helped |
| Select synthetic examples with the filter instead of randomly (ablation) | LS-PT augmentation · AAPD CG | accuracy: 29.32 -> 30.21 | Helped |
| Select synthetic examples with the filter instead of randomly (ablation) | LD-VAE augmentation · AAPD CG | accuracy: 28.82 -> 31.11 | Helped |
| Increase classifier support examples from 50 to 250 (ablation) | BERT with LD-VAE augmentation · AAPD CG | accuracy: 32.09 -> 35.23 | Helped |
| Increase conditional text-generator support examples from 50 to 250 (ablation) | LD-VAE augmentation · AAPD CG | accuracy: 32.09 -> 31.92 | No clear effect |
Surprising findings
- T5+CLP, which is state-of-the-art on standard MLTC splits, performs the worst among baselines on the CG split.
- BERT+P (parameter-efficient tuning) does not outperform fine-tuned BERT on MLTC CG, contrary to prior semantic parsing findings.
Limitations
- Generated examples sometimes reflect only a subset of the target label composition even after filtering.
- Support set is tiny (50 examples) and zero-shot label generation performs notably worse.
- Experiments limited to three English benchmarks with BERT-base and GPT-2 scale models.
Experimental setup
Datasets: SemEval AAPD IMDB
Benchmarks: SemEval AAPD IMDB
Metrics: Jaccard Accuracy Correctness Completeness
Baselines: BERT BERT+P BERT+DBloss BERT+MAGNET BERT+SGM T5+CLP Concat Flan-T5 VAE-DPrior GPT3.5 GPT2-PT
Builds on: VAE-DPrior prefix-tuning p-tuning GPT-2 BERT
Closest papers by meaning
- Label-Specific Feature Augmentation for Long-Tailed Multi-Label Text Classification (AAAI 2023 · similarity 0.87)
- Consistency of Compositional Generalization Across Multiple Levels (AAAI 2025 · similarity 0.86)
- Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions (ICLR 2024 · similarity 0.86)
- Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classification (AAAI 2024 · similarity 0.85)
- Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification (ICLR 2025 · similarity 0.85)
- Generalization Analysis for Multi-Label Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.