Research map / Learning theory and algorithms / Clustering
Breaking the Reclustering Barrier in Centroid-based Deep Clustering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies a "reclustering barrier" in centroid-based deep clustering (DEC, IDEC, DCN) and proposes BRB, which combines soft encoder weight resets with k-Means reclustering and momentum resets to escape performance plateaus.
Problem
Centroid-based deep clustering performance saturates early, and periodic reclustering with k-Means fails to yield late-training improvements because the embedding does not change enough.
Approach
Apply soft weight resets (shrink-and-perturb style convex combination of current and freshly initialized weights) to the encoder, then recluster the perturbed embedding with k-Means and reset centroid momentum, creating structural embedding changes that let reclustering explore new clustering solutions.
Key results
- BRB improves performance in 88.10% of all runs with approximately 1.1% runtime overhead.
- With contrastive learning and self-labeling on CIFAR100-20, DCN+BRB reaches 56.92% ACC, exceeding SeCu's 55.2%.
- Without pre-training, BRB improves DEC on OPTDIGITS from 61 to 77 accuracy without augmentation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BRB to DEC | DEC with contrastive learning and self-labeling · CIFAR10 | ACC: 88.29 -> 90.57 | Helped |
| Add BRB to DEC | DEC with contrastive learning and self-labeling · CIFAR10 | NMI: 80.60 -> 82.57 | Helped |
| Add BRB to DEC | DEC with contrastive learning and self-labeling · CIFAR10 | ARI: 77.23 -> 81.18 | Helped |
| Add BRB to IDEC | IDEC with contrastive learning and self-labeling · CIFAR10 | ACC: 88.30 -> 90.72 | Helped |
| Add BRB to IDEC | IDEC with contrastive learning and self-labeling · CIFAR10 | NMI: 79.50 -> 83.26 | Helped |
| Add BRB to IDEC | IDEC with contrastive learning and self-labeling · CIFAR10 | ARI: 77.27 -> 81.81 | Helped |
| Add BRB to DCN | DCN with contrastive learning and self-labeling · CIFAR10 | ACC: 88.55 -> 91.23 | Helped |
| Add BRB to DCN | DCN with contrastive learning and self-labeling · CIFAR10 | NMI: 81.02 -> 83.66 | Helped |
| Add BRB to DCN | DCN with contrastive learning and self-labeling · CIFAR10 | ARI: 78.17 -> 82.42 | Helped |
| Add BRB to DEC | DEC with contrastive learning and self-labeling · CIFAR100-20 | ACC: 50.16 -> 50.46 | Helped |
| Add BRB to DEC | DEC with contrastive learning and self-labeling · CIFAR100-20 | NMI: 51.66 -> 51.72 | Helped |
| Add BRB to DEC | DEC with contrastive learning and self-labeling · CIFAR100-20 | ARI: 35.37 -> 35.05 | Hurt |
Surprising findings
- Reclustering alone barely changes the embedding or cluster labels, so periodic k-Means reclustering cannot break late-training plateaus.
- Resetting the final embedding layer or using non-persistent noise hurts performance, indicating the soft reset structure matters.
Limitations
- Analysis limited to centroid-based deep clustering algorithms; non-centroid methods not studied.
- No full theoretical analysis of BRB; connection to perturbation-resilient clustering is hypothesized only.
- Hyperparameter sensitivity: frequent or strong resets destroy cluster structure and performance depends on dataset.
Experimental setup
Datasets: MNIST KMNIST FMNIST USPS OPTDIGITS GTSRB CIFAR10 CIFAR100-20
Benchmarks: MNIST KMNIST FMNIST USPS OPTDIGITS GTSRB CIFAR10 CIFAR100-20
Metrics: accuracy ARI NMI inter-class distance intra-class distance cluster label change
Baselines: DEC IDEC DCN SCAN GCC SeCu
Builds on: DEC IDEC DCN SimCLR Shrink and Perturb
Closest papers by meaning
- You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering (NeurIPS 2025 · similarity 0.86)
- Proportional Fairness in Non-Centroid Clustering (NeurIPS 2024 · similarity 0.86)
- Efficient Constrained K-center Clustering with Background Knowledge (AAAI 2024 · similarity 0.85)
- Clustering via Self-Supervised Diffusion (ICML 2025 · similarity 0.85)
- Hierarchical Overlapping Clustering on Graphs: Cost Function, Algorithm and Scalability (ICML 2025 · similarity 0.85)
- THESAURUS: Contrastive Graph Clustering by Swapping Fused Gromov-Wasserstein Couplings (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.