Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Calibrated Deep Clustering Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a calibrated deep clustering network (CDC) with a dual-head design, where a calibration head penalizes overconfident predictions in unreliable K-means regions and selects pseudo-labels for the clustering head, plus a prototype-based head initialization.
Problem
Deep clustering models are severely overconfident: predicted confidence greatly exceeds actual accuracy, and existing calibration methods need labeled validation data or hurt pseudo-label quality.
Approach
A calibration head is trained to match the averaged clustering-head predictions within K-means mini-clusters (plus an entropy term), penalizing confidence only in unreliable regions; its calibrated confidence then drives class-dynamic pseudo-label selection for the clustering head, and both heads are initialized by K-means prototypes on features.
Key results
- CDC-Cal reaches ECE of 1.1% on CIFAR-10 and 4.9% on CIFAR-20 versus 15.4% and 52.3% for SPICE-2 (table values).
- CDC-Cal achieves 61.7% ACC on CIFAR-20 and 93.0% on STL-10, ranking first in 11 of 12 ACC/ARI cases per the authors.
- Prototype-based initialization raises CIFAR-20 ACC of the clustering head from 10.4% (random init) to 56.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · CIFAR-10 | ACC: 94.3 -> 94.9 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · CIFAR-20 | ACC: 61.4 -> 61.7 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · STL-10 | ACC: 92.6 -> 93.0 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · ImageNet-10 | ACC: 97.0 -> 97.3 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · Tiny-ImageNet | ACC: 30.8 -> 33.9 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · CIFAR-10 | ECE: 4.9 -> 1.1 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · CIFAR-20 | ECE: 13.1 -> 4.9 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · STL-10 | ECE: 6.5 -> 0.9 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · ImageNet-10 | ECE: 1.5 -> 0.8 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · ImageNet-Dogs | ECE: 19.3 -> 7.7 | Helped |
| use CDC-Cal instead of the strongest listed non-CDC baseline | deep clustering · Tiny-ImageNet | ECE: 3.2 -> 11.0 | Hurt |
| add prototype-based initialization (ablation) | CDC · CIFAR-20 | ACC: 44.4 -> 61.7 | Helped |
Surprising findings
- Label smoothing and focal loss over-penalize high-confidence predictions, degrading failure rejection, while Focal Loss gets lower ECE but much worse ACC.
- Single-head designs that let the overconfident head select pseudo-labels hurt both clustering and calibration.
Limitations
- Requires choosing the number of mini-clusters K per dataset, with guidance only empirical (B/K ratios).
- Theoretical guarantees rely on a binary Gaussian mixture setting and 'mild conditions'.
- Some baselines results are cited from papers rather than re-run under identical settings.
Experimental setup
Datasets: CIFAR-10 CIFAR-20 STL-10 ImageNet-10 ImageNet-Dogs Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-20 STL-10 ImageNet-10 ImageNet-Dogs Tiny-ImageNet
Metrics: ACC NMI ARI ECE AUROC AURC FPR95
Baselines: K-means MoCo-v2 SimSiam BYOL DMICC ProPos CoNR DivClust CC TCC TCL SeCu
Builds on: MoCo-v2 SCAN SPICE Temperature Scaling Label Smoothing
Closest papers by meaning
- Learning from Sample Stability for Deep Clustering (ICML 2025 · similarity 0.86)
- You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering (NeurIPS 2025 · similarity 0.86)
- A Benchmark Study on Calibration (ICLR 2024 · similarity 0.84)
- Reassessing How to Compare and Improve the Calibration of Machine Learning Models (ICLR 2025 · similarity 0.84)
- From Logits to Hierarchies: Hierarchical Clustering made Simple (ICML 2025 · similarity 0.84)
- Hard Regularization to Prevent Deep Online Clustering Collapse without Data Augmentation (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.