Research map / Label-efficient and robust learning / Recognition and long-tail
A Unified Knowledge Transfer Network for Generalized Category Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Knowledge Transfer Network (KTN) for text generalized category discovery, using entropy-based soft differentiation of transfer candidates and prototype-based transfer weights to adjust logits and pseudo-labels from known to novel categories.
Problem
Generalized Category Discovery in text: recognizing known and novel categories in unlabeled data, where novel categories get poor performance due to lack of supervision and knowledge transfer from known categories is hindered by ambiguous transfer candidates and uncertain transfer weights.
Approach
Identify novel-category samples via the entropy of a frozen pre-trained known-class classifier (soft sigmoid mapping of entropy deviation from batch mean), derive transfer weights from cosine similarities between known and novel category prototypes, and adjust logits and post-hoc pseudo-labels (via Sinkhorn-Knopp) for transfer candidates accordingly; a unified prototype classifier enables fast onl
Key results
- KTN achieves 77.93% All accuracy on BANKING, 81.32% on HWU64 and 91.28% on CLINC, outperforming DPN across metrics.
- Novel-category accuracy improves by 11.58, 11.43 and 4.3 points over the SOTA on BANKING, HWU64 and CLINC respectively.
- Inference with KTN takes 3.12s on BANKING versus 17.68s for DAC, enabling online deployment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add KTN knowledge transfer | BERT-base-uncased · BANKING | Novel accuracy: 48.60 -> 60.18 | Helped |
| add KTN knowledge transfer | BERT-base-uncased · HWU64 | Novel accuracy: 60.64 -> 72.07 | Helped |
| add KTN knowledge transfer | BERT-base-uncased · CLINC | Novel accuracy: 77.54 -> 81.84 | Helped |
| add infoNCE loss Lcl (ablation) | KTN · BANKING | All accuracy: 77.48 -> 77.93 | Helped |
| add swapped prediction (ablation) | KTN · BANKING | All accuracy: 77.29 -> 77.93 | Helped |
| add CE loss Lcel on labeled data (ablation) | KTN · BANKING | Known accuracy: 75.03 -> 83.85 | Helped |
| add classifier initialization (ablation) | KTN · BANKING | Novel accuracy: 42.47 -> 60.18 | Helped |
| add label adjustment (ablation) | KTN · BANKING | Novel accuracy: 35.71 -> 60.18 | Helped |
| add logits-level adjustment (ablation) | KTN label adjustment · BANKING | Novel accuracy: 52.12 -> 60.18 | Helped |
| add label-level adjustment (ablation) | KTN label adjustment · BANKING | Novel accuracy: 45.13 -> 60.18 | Helped |
| use ESD to identify transfer candidates and prototype-based transfer weights instead of all candidates and uniform weights (ablation) | KTN label adjustment · BANKING | All accuracy: 73.21 -> 77.93 | Helped |
| use prototype-based transfer weights instead of uniform weights (ablation) | KTN label adjustment · BANKING | Novel accuracy: 65.12 -> 60.18 | Helped |
Surprising findings
- Removing label adjustment drops novel-category accuracy to 35.71% on BANKING while known accuracy slightly rises, indicating a trade-off between known and novel performance.
- Removing CE loss on labeled data gives higher novel accuracy (65.48%) than the full model but much lower known accuracy (75.03%).
Limitations
- Requires prior knowledge of the total number of categories K, as assumed by prior work.
- Relies on the pre-trained classifier's entropy being biased toward known categories, and hyperparameters (alpha, gamma, m) are set manually.
- Evaluated only on three intent-classification text benchmarks.
Experimental setup
Datasets: BANKING CLINC HWU64
Benchmarks: BANKING HWU64 CLINC
Metrics: accuracy on known categories clustering accuracy on novel categories overall accuracy (All)
Baselines: DeepCluster DCN DEC GloVe-KM SAE-KM Semi-DC CDAC+ DTC Semi-KM DAC GCD DPN
Builds on: DPN DAC GCD (Vaze et al.) SwAV-style Sinkhorn-Knopp Zhang et al. 2022 pretraining
Closest papers by meaning
- Transfer and Alignment Network for Generalized Category Discovery (AAAI 2024 · similarity 0.95)
- Prior-Constrained Association Learning for Fine-Grained Generalized Category Discovery (AAAI 2025 · similarity 0.90)
- Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective (NeurIPS 2025 · similarity 0.88)
- Joint Class-level and Instance-level Relationship Modeling for Novel Class Discovery (AAAI 2025 · similarity 0.88)
- HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts (ICLR 2025 · similarity 0.88)
- Active Generalized Category Discovery (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.