Research map / Label-efficient and robust learning / Recognition and long-tail

Learning from Sample Stability for Deep Clustering

ICML 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Correspondence and causal representation learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LFSS, a deep clustering method that uses per-sample representation stability across consecutive epochs (via a predecessor network) as instance- and cluster-level supervision alongside BYOL-style self-distillation.

Problem

Deep clustering relies on supervision signals (reconstruction, similarity, pseudo-labels), but existing methods ignore the training progress of individual samples, and unstable samples are often misclustered.

Approach

Define sample stability as the cosine similarity of a sample's representation across consecutive epochs, observe that unstable samples are mispredicted and hard to memorize, and use stability as a supervision signal: a contrastive instance loss between current and predecessor-epoch representations, plus a cluster-center contrastive loss computed after excluding the most unstable samples, combined

Key results

• LFSS reaches 93.4% ACC on CIFAR-10 and 86.1% on STL-10, outperforming the best listed baselines (e.g., ProPos 92.0% on CIFAR-10).
• Compared with the BYOL baseline, LFSS improves ACC by 10.1 points on CIFAR-10 (83.3 to 93.4) and 5.4 to 18.6 points across all metrics and datasets.
• Ablation on CIFAR-10 shows removing instance-level loss drops ACC from 93.4% to 89.6%, and removing cluster-level loss drops it to 89.5%.

Measured effects reported in the paper

Surprising findings

• Samples that are unstable across epochs recur as unstable over multiple epochs far more often than chance, suggesting long-term instability is a property of the samples rather than the network state.
• Introducing the cluster-level loss too early (eta = 0 or 50) degrades performance because early cluster centers are of low quality.

Limitations

• Relies on the number of clusters K being known a priori for K-means and cluster-level loss.
• Warm-up schedule and hyper-parameters (eta, sigma, lambda, delta) are dataset-dependent and tuned per dataset.
• Reported results are averages over 10 K-means runs on reproduced baselines with a single backbone setup, with large-scale results only on fewer epochs (50 epochs on ImageNet-1K).

Experimental setup

Datasets: CIFAR-10 CIFAR-20 STL-10 ImageNet-10 ImageNet-Dogs Tiny-ImageNet ImageNet-1K

Benchmarks: CIFAR-10 CIFAR-20 STL-10 ImageNet-10 ImageNet-Dogs Tiny-ImageNet ImageNet-1K

Metrics: clustering accuracy (ACC) normalized mutual information (NMI) adjusted rand index (ARI)

Baselines: IIC DCCM PICA SCAN NMM CC MiCE GCC PCL TCC IDFD ProPos

Builds on: BYOL SimCLR ProPos NT-Xent loss

Closest papers by meaning

• Towards Calibrated Deep Clustering Network (ICLR 2025 · similarity 0.86)
• Towards Memorization Estimation: Fast, Formal and Free (ICML 2025 · similarity 0.86)
• P$^2$OT: Progressive Partial Optimal Transport for Deep Imbalanced Clustering (ICLR 2024 · similarity 0.86)
• You Can Trust Your Clustering Model: A Parameter-free Self-Boosting Plug-in for Deep Clustering (NeurIPS 2025 · similarity 0.85)
• On the Stability of Graph Convolutional Neural Networks: A Probabilistic Perspective (NeurIPS 2025 · similarity 0.84)
• Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR 2024 · similarity 0.84)