Research map / Label-efficient and robust learning / Recognition and long-tail

Unleashing the Potential of Model Bias for Generalized Category Discovery

AAAI 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Data augmentation and generalization bias · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Self-Debiasing Calibration (SDC), which uses a frozen biased BERT classifier trained on known categories to debias logits (category bias) and transfer knowledge between similar categories (category confusion) for generalized category discovery in text.

Problem

Generalized category discovery in text: identifying known and novel categories in unlabeled data, where a model pre-trained only on known categories is biased toward known classes and novel categories are confused with each other.

Approach

Rather than treating the known-category bias of a pre-trained model as purely harmful, the frozen biased model's logits are used to measure and subtract the bias from the training model's known-category logits, and to add transfer logits (via a prototype-similarity matrix) to novel-category logits; per-instance entropy of the biased output weights the adjustment to avoid penalizing known-category

Key results

• SDC reaches H-score 70.11 on BANKING, 79.81 on HWU64, and 87.65 on CLINC, outperforming the best prior method KTN (68.20, 76.34, 86.13).
• Average novel-category accuracy improves by 2.64% over SOTA and average H-score by 2.00%, per the authors.
• Ablation on HWU64 shows removing the MLM pre-training loss drops H-score from 79.81 to 74.73, and removing CBM drops it to 77.78.

Measured effects reported in the paper

Surprising findings

• GPT-3.5 and GPT-4 achieve high known-category accuracy but very low novel-category accuracy (e.g., GPT-4 novel 15.44 on CLINC), showing LLMs struggle with novel categories without prior knowledge.

Limitations

• Evaluated only on three text intent-detection datasets; generalization to other domains or images is not shown.
• Assumes the total number of categories K is known, with estimation handled as a separate step.
• Relies on a biased pre-trained model whose quality governs the debiasing signal.

Experimental setup

Datasets: BANKING HWU64 CLINC

Benchmarks: BANKING HWU64 CLINC

Metrics: clustering accuracy Known accuracy Novel accuracy H-score

Baselines: DeepCluster DCN DEC BERT GloVe SAE Simple Semi-DC CDAC+ DTC Semi-KM DAC

Builds on: BERT Sinkhorn-Knopp DTC GCD

Closest papers by meaning

• Transfer and Alignment Network for Generalized Category Discovery (AAAI 2024 · similarity 0.89)
• DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery (ICLR 2025 · similarity 0.87)
• Happy: A Debiased Learning Framework for Continual Generalized Category Discovery (NeurIPS 2024 · similarity 0.86)
• A Unified Knowledge Transfer Network for Generalized Category Discovery (AAAI 2024 · similarity 0.86)
• Prior-Constrained Association Learning for Fine-Grained Generalized Category Discovery (AAAI 2025 · similarity 0.86)
• Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective (NeurIPS 2025 · similarity 0.86)