Research map / Continual learning / Class-incremental
Black-Box Forgetting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Black-Box Forgetting, which makes a black-box CLIP forget specified classes by optimizing a text prompt with CMA-ES, using a Latent Context Sharing parametrization of prompt tokens.
Problem
Selectively forgetting specified classes in large pre-trained vision-language models when only black-box access (no weights or gradients) is available, while keeping accuracy on the remaining classes.
Approach
Optimize a learnable text prompt with derivative-free CMA-ES using an entropy-maximization loss on forgotten classes and cross-entropy on memorized classes; reduce dimensionality with Latent Context Sharing, where each context mixes a shared low-dimensional component with unique per-token components.
Key results
- On CIFAR-10, Ours reaches H=95.07 with Err_for 96.10 and Acc_mem 94.06, versus BBT H=85.69 and CBBT H=93.48.
- On ImageNet30, Ours reaches H=97.28 versus CBBT 87.88 and BBT 94.22, and is close to white-box CoOp (99.30).
- Ours (w/o LCS) underperforms the full method on all four benchmarks, e.g. H=72.37 vs 95.07 on CIFAR-10, supporting the shared-component idea.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Latent Context Sharing (LCS) instead of optimizing latent contexts independently (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · CIFAR-10 | H: 72.37_{\pm 0.13} -> 95.07_{\pm 0.01} | Helped |
| add Latent Context Sharing (LCS) instead of optimizing latent contexts independently (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · CIFAR-100 | H: 79.38_{\pm 0.02} -> 80.99_{\pm 0.01} | Helped |
| add Latent Context Sharing (LCS) instead of optimizing latent contexts independently (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · CUB-200-2011 | H: 58.78_{\pm 0.01} -> 59.67_{\pm 0.01} | Helped |
| add Latent Context Sharing (LCS) instead of optimizing latent contexts independently (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · ImageNet30 | H: 95.26_{\pm 0.02} -> 97.28_{\pm 0.01} | Helped |
| add Latent Context Sharing (LCS) instead of optimizing latent contexts independently (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · ImageNet-1k | H: 74.65_{\pm 0.00} -> 76.35_{\pm 0.00} | Helped |
| use the proposed black-box prompt forgetting method rather than zero-shot CLIP | CLIP ViT-B/16 · CIFAR-10 | H: 15.30 -> 95.07_{\pm 0.01} | Helped |
| prioritize memorized-class accuracy in the loss (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · CUB-200-2011 | Acc_mem: 44.81_{\pm 0.01} -> 46.65_{\pm 0.01} | Helped |
| prioritize memorized-class accuracy in the loss (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · CUB-200-2011 | Err_for: 89.29_{\pm 0.01} -> 80.87_{\pm 0.00} | Hurt |
| combine few-shot forgetting with C-Emb. for classes without training samples (ablation) | Black-Box Forgetting · ImageNet30 | H: 89.34_{\pm 0.03} -> 92.30_{\pm 0.05} | Helped |
| use class-embedding-only zero-shot tuning instead of few-shot prompt tuning (ablation) | Black-Box Forgetting · CIFAR-100 | H: 80.99_{\pm 0.01} -> 64.00_{\pm 0.01} | Hurt |
| sample the random projection from N(0,σ) rather than N(0,1) (ablation) | Black-Box Forgetting with CLIP ViT-B/16 · CIFAR-10 | H: 84.58_{\pm 0.05} -> 95.07_{\pm 0.01} | Helped |
| update model parameters while tuning contexts with CoOp (ablation) | CoOp (White-Box) · CIFAR-10 | H: 96.49_{\pm 0.00} -> 75.74_{\pm 0.24} | Hurt |
Surprising findings
- Tuning with only class embeddings (zero-shot C-Emb.) gives much lower memorized-class accuracy than the few-shot method, e.g. 85.61 vs 94.06 on CIFAR-10.
- Simultaneously updating CLIP parameters in white-box CoOp hurts forgetting performance compared with prompt-only tuning.
Limitations
- Assumes access to the context (token) embeddings of the target model, so it is not fully black-box.
- Lower memorized-class accuracy (Acc_mem) than baselines on several datasets due to a forgetting trade-off.
- Few-shot setting; zero-shot class-embedding variant performs poorly.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CUB-200-2011 ImageNet30 ImageNet-1k
Benchmarks: CIFAR-10 CIFAR-100 CUB-200-2011 ImageNet30 ImageNet-1k
Metrics: Err_for Acc_mem harmonic mean H
Baselines: Zero-Shot CLIP BBT CBBT CoOp Sep-CMA-ES VkD-CMA-ES
Builds on: Black-Box Tuning (BBT) CoOp CMA-ES Learning with Selective Forgetting
Closest papers by meaning
- Targeted Forgetting of Image Subgroups in CLIP Models (CVPR 2025 · similarity 0.86)
- Reviewing the Forgotten Classes for Domain Adaptation of Black-Box Predictors (AAAI 2024 · similarity 0.86)
- MOS: Model Surgery for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.86)
- Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning (NeurIPS 2024 · similarity 0.86)
- Few-Shot Learning via Repurposing Ensemble of Black-Box Models (AAAI 2024 · similarity 0.85)
- Query Efficient Black-Box Visual Prompting with Subspace Learning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.