Research map / Probabilistic and scientific modeling / Molecules and proteins
From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RR-CLARC, which post-hoc unlearns latent concept biases by explicitly penalizing the output gradient along a signal-CAV direction, and shows that regression-based CAVs (e.g., SVM) diverge from the true concept direction.
Problem
Deep networks learn spurious correlations (Clever Hans shortcuts) that bias predictions; existing post-hoc correction either needs pixel-level annotations (only for localized biases) or only augments latent activations without explicitly removing bias sensitivity. Also, CAVs from regression solvers
Approach
Compute a robust bias direction with a correlation-based signal-CAV, then fine-tune with a loss that squares the latent gradient of the (optionally class-weighted) output along that direction, explicitly reducing bias sensitivity; the feature extractor up to the CAV layer is frozen so the direction stays fixed.
Key results
- RR-CLARC achieves the best biased-test accuracy in most settings, e.g., VGG-16 ISIC biased accuracy 57.0% vs 42.8% for RRR and 34.9% for Vanilla, with TCAV 0.49.
- Signal-CAVs reach overall alignment above 90% for VGG-16 versus under 60% for regression-based CAVs.
- On CelebA with VGG-16, RR-CLARC raises biased accuracy to 92.6% (Vanilla 82.8%) with TCAV 0.54, while RRR and ACLARC are less effective on unlocalized artifacts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RRCLARC instead of the vanilla model | VGG-16 · Bone Age biased test set | accuracy: 49.8 -> 77.7 | Helped |
| Use RRCLARC instead of the vanilla model | VGG-16 · ISIC biased test set | accuracy: 34.9 -> 57.0 | Helped |
| Use RRCLARC instead of the vanilla model | VGG-16 · ImageNet biased test set | accuracy: 43.5 -> 62.6 | Helped |
| Use RRCLARC instead of the vanilla model | VGG-16 · CelebA biased test set | accuracy: 82.8 -> 92.6 | Helped |
| Use signal-CAV instead of regression-based CAV optimizers (ablation) | RRCLARC · Controlled bias experiments | accuracy on biased test set | Helped |
| Increase regularization strength λ (ablation) | RRCLARC · Biased test sets | accuracy | Mixed |
| Penalize only the selected class rather than all classes (ablation) | RRCLARC · ImageNet clean test set, selected related classes | accuracy | Helped |
| Apply activation-based A-CLARC or P-CLARC rather than class-specific gradient penalization | ImageNet correction · ImageNet clean test set, selected related classes | accuracy | Hurt |
| Use input-gradient RRR for unlocalized artifacts | VGG-16 · Bone Age biased test set | accuracy: 49.8 -> 49.8 | No clear effect |
Surprising findings
- Regression-based CAVs (SVM, ridge, lasso, logistic) diverge substantially from the true concept direction, even though their localization heatmaps can look sensible.
- RRR fails to reduce bias on unlocalized artifacts (ISIC, Bone Age), while it works for localized ones.
Limitations
- Requires freezing all layers up to the CAV layer during fine-tuning.
- Performance depends on a well-aligned CAV; clean accuracy can drop when the bias is entangled with useful features.
- Choice of the last convolutional layer for CAV computation may not be optimal; optimal layer selection is open.
Experimental setup
Datasets: ISIC2019 Pediatric Bone Age ImageNet CelebA
Benchmarks: ISIC2019 Pediatric Bone Age ImageNet CelebA
Metrics: accuracy on clean test set accuracy on biased test set TCAV score CAV alignment cosine similarity R_bias relevance
Baselines: Vanilla RRR PCLARC ACLARC
Builds on: CLARC Right for the Right Reasons (RRR) TCAV signal-CAV
Closest papers by meaning
- Controlling for Omitted Variable Bias in Deep Neural Networks (NeurIPS 2026 · similarity 0.86)
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.85)
- NeuronTune: Towards Self-Guided Spurious Bias Mitigation (ICML 2025 · similarity 0.85)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.85)
- Discovering Bias in Latent Space: An Unsupervised Debiasing Approach (ICML 2024 · similarity 0.85)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.