Research map / Label-efficient and robust learning / Recognition and long-tail
Enhancing Multi-Label Classification via Dynamic Label-Order Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes dynamic label-order learning (DLOL) for generative multi-label classification, choosing per-sample label orders by perplexity-based difficulty with a jointly optimized BART-base seq2seq model.
Problem
Generative (seq2seq) multi-label classification depends heavily on label order; static orderings memorize training order and cause missed labels at inference.
Approach
Adaptively learns a per-sample label order using a difficulty-prioritized principle: iteratively selects the unordered label with highest perplexity under the same seq2seq model, and jointly optimizes label-order learning and MLC with a unified multi-reference plus label-smoothing loss and an EOS penalty at decoding.
Key results
- DLOL reaches average F1 of 59.2 across four datasets, versus 57.2 for the best prior generative baseline (BART+SETAUG) and 56.5 for the best discriminative baseline (BERT+LACO).
- On Reuters-21578 DLOL achieves 68.8 F1 vs 65.9 for BART+SETAUG; on GoEmotions 53.8 vs 51.7.
- Using difficulty-prioritized ordering in prompted GPT-3.5-Turbo raises GoEmotions F1 from 37.6 (random order) to 42.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the combined multi-reference loss instead of Vinyals loss (ablation) | DLOL · Reuters-21578 | F1: 67.5 -> 68.8 | Helped |
| use the combined multi-reference loss instead of Qin loss (ablation) | DLOL · Reuters-21578 | F1: 67.7 -> 68.8 | Helped |
| add label smoothing (ablation) | DLOL · Reuters-21578 | F1: 67.9 -> 68.8 | Helped |
| add EOS penalty (ablation) | DLOL · Reuters-21578 | F1: 68.3 -> 68.8 | Helped |
| jointly optimize label-order learning and MLC learning instead of separate optimization (ablation) | DLOL · Reuters-21578 | F1: 67.4 -> 68.8 | Helped |
| add label-order optimization (ablation) | DLOL · Reuters-21578 | F1: 66.9 -> 68.8 | Helped |
| use dynamic difficulty-prioritized ordering instead of alphabetical ordering (ablation) | DLOL · Reuters-21578 | F1: 65.2 -> 68.8 | Helped |
| use dynamic difficulty-prioritized ordering instead of frequency-first ordering (ablation) | DLOL · Reuters-21578 | F1: 65.1 -> 68.8 | Helped |
| use dynamic difficulty-prioritized ordering instead of ease-prioritized ordering (ablation) | DLOL · Reuters-21578 | F1: 66.0 -> 68.8 | Helped |
| use dynamic rather than development-set-based static difficulty prioritization (ablation) | DLOL · Reuters-21578 | F1: 67.1 -> 68.8 | Helped |
| prioritize difficult labels over uniformly random label order (ablation) | BART-base on Reuters-21578 · Reuters-21578 | F1: +1.4 | Helped |
| prioritize difficult labels over ease-prioritized ordering (ablation) | BART-base on Reuters-21578 · Reuters-21578 | F1: 66.1 vs 62.8 | Helped |
Surprising findings
- Static label orders yield only about 1% of test predictions differing from the training order, indicating rote memorization of order and lower recall.
- Ease-prioritized ordering hurts overall performance, while difficulty-prioritized ordering improves it.
Limitations
- Label-order learning cost grows cubically with the number of labels per sample.
- Performance is sensitive to the hyperparameters alpha, beta, and gamma, requiring tuning.
- LLM experiments use gold labels with prompts to sort by difficulty, since perplexity is infeasible for these models; evaluated on only one dataset.
Experimental setup
Datasets: Reuters-21578 RCV1-V2 Slashdot GoEmotions
Benchmarks: Reuters-21578 RCV1-V2 Slashdot GoEmotions
Metrics: macro precision macro recall macro F1
Baselines: BERT ML-REASONER CORE BOX HIDDEN NPCRF LACO SEQ2SET OTSEQ2SET BART SGM SETRNN
Builds on: SGM SETRNN SETAUG Vinyals et al. 2016 order matters loss Qin et al. 2019 multi-reference loss
Closest papers by meaning
- Learning Context-dependent Label Permutations for Multi-label Classification (ICML 2019 · similarity 0.91)
- Decouple then Classify: A Dynamic Multi-view Labeling Strategy with Shared and Specific Information (ICML 2024 · similarity 0.87)
- Predicting Label Distribution from Ternary Labels (NeurIPS 2024 · similarity 0.86)
- Batch Selection for Multi-Label Classification Guided by Uncertainty and Dynamic Label Correlations (AAAI 2025 · similarity 0.86)
- Addressing Multi-Label Learning with Partial Labels: From Sample Selection to Label Selection (AAAI 2025 · similarity 0.86)
- Self-Paced Unified Representation Learning for Hierarchical Multi-Label Classification (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.