Research map / Label-efficient and robust learning / Recognition and long-tail
CoUn: Empowering Machine Unlearning via Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoUn, a machine unlearning method that applies contrastive learning and supervised cross-entropy only on retain data so forget representations drift toward semantically similar retain clusters, outperforming baselines on CIFAR-10/100 and TinyImageNet.
Problem
Approximate machine unlearning struggles to close the gap with exact unlearning (retraining from scratch): label-manipulation methods hurt utility, and weight-perturbation methods trade off forget quality against utility, especially without access to forget data.
Approach
Observing that a retrain model classifies forget samples by semantic similarity to retain clusters, CoUn applies self-supervised InfoNCE contrastive learning plus cross-entropy on retain data only; cluster collisions among retain samples indirectly push forget representations toward semantically similar retain clusters.
Key results
- With ResNet-18 at 10% random forgetting, CoUn achieves the lowest average gap to Retrain on CIFAR-10 (0.25), CIFAR-100 (1.39) and TinyImageNet (1.95), per the paper's table.
- Integrating CoUn's CL module into NegGrad+, l1-sparse and NoT on CIFAR-10 ResNet-18 improves them by 44.8%, 42.1% and 43.3% at 10% forgetting.
- At 50% random forgetting, CoUn gets average gap 0.40 on CIFAR-10 ResNet-18 versus 0.90 for the best baseline (NegGrad+).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CoUn instead of the strongest listed baseline | ResNet-18 · CIFAR-10, 10% random removal | average gap: 0.30 -> 0.25 | Helped |
| Use CoUn instead of the strongest listed baseline | ResNet-18 · CIFAR-100, 10% random removal | average gap: 1.65 -> 1.39 | Helped |
| Use CoUn instead of the strongest listed baseline | ResNet-18 · TinyImageNet, 10% random removal | average gap: 2.08 -> 1.95 | Helped |
| Use CoUn instead of the strongest listed baseline | VGG-16 · CIFAR-100, 10% random removal | average gap: 2.81 -> 1.31 | Helped |
| Use CoUn instead of the strongest listed baseline | ViT · CIFAR-100, 10% random removal | average gap: 3.98 -> 3.80 | Helped |
| Use CoUn instead of the strongest listed baseline | ResNet-18 · CIFAR-10, 50% random removal | average gap: 0.90 -> 0.40 | Helped |
| Use CoUn instead of the strongest listed baseline | ResNet-18 · CIFAR-100, 50% random removal | average gap: 2.11 -> 1.30 | Helped |
| Use CoUn instead of the strongest listed baseline | ResNet-18 · TinyImageNet, 50% random removal | average gap: 2.01 -> 1.59 | Helped |
| Use CoUn instead of the strongest listed baseline | VGG-16 · CIFAR-100, 50% random removal | average gap: 4.24 -> 0.85 | Helped |
| Use CoUn instead of the strongest listed baseline | ViT · CIFAR-100, 50% random removal | average gap: 4.86 -> 3.91 | Helped |
| Set CL temperature to 0.1 rather than deviating from it (ablation) | CoUn · CIFAR-100, 50% random removal | average gap | Helped |
| Use matching supervised-learning and CL transformations (CHN) rather than stronger or overly simple transformations (ablation) | CoUn · CIFAR-100, 50% random removal | average gap | Helped |
Surprising findings
- Under class-wise forgetting, plain fine-tuning performs comparably to Retrain, since forget and retain data are weakly entangled in that setting.
- Stronger CL augmentations than those used for supervised learning (e.g., CHJGN) degrade performance, attributed to tighter clustering of forget representations.
Limitations
- Hyperparameters (lambda, tau, augmentations) require careful tuning; performance is sensitive to them.
- Evaluated only on image classification with random and class-wise forgetting on small-scale datasets and three architectures.
- The theoretical analysis relies on assumptions (Lipschitz feature extractor, (sigma, delta)-augmentation) and the Lemma depends on an assumed flat-minima argument not guaranteed in general.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet
Metrics: retain accuracy unlearn accuracy test accuracy membership inference attack average gap to Retrain PFLOPs
Baselines: Retrain FT NegGrad+ l1-sparse SalUn NoT BadT SSD SCRUB CU
Builds on: SimCLR SalUn NegGrad+ NoT SCRUB
Closest papers by meaning
- Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance (NeurIPS 2026 · similarity 0.89)
- Label-Agnostic Forgetting: A Supervision-Free Unlearning in Deep Models (ICLR 2024 · similarity 0.89)
- Contrastive Representation Shaping for LLM Unlearning (NeurIPS 2026 · similarity 0.88)
- Fast Machine Unlearning without Retraining through Selective Synaptic Dampening (AAAI 2024 · similarity 0.88)
- When to Forget? Complexity Trade-offs in Machine Unlearning (ICML 2025 · similarity 0.88)
- What to Forget in Unlearning? Forget Set Curation for Language Models (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.