Research map / Federated learning and privacy / Privacy attacks and secure learning
Ferrari: Federated Feature Unlearning via Optimizing Feature Sensitivity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Ferrari, a client-side federated feature unlearning method that minimizes a Lipschitz-based feature sensitivity measure using only the unlearning client's local data.
Problem
Feature unlearning in federated learning (removing sensitive, backdoor, or biased features) existing methods need other clients' data, and there is no good metric to evaluate feature unlearning effectiveness.
Approach
Defines feature sensitivity as the expected output change rate under random perturbations of the feature region (Lipschitz-inspired), and unlearns by minimizing this quantity on the unlearning client's local data without requiring other clients.
Key results
- On backdoor unlearning (MNIST), Ferrari reduces backdoor-set accuracy Du to 0.11% while keeping Dr at 95.93%, close to Retrain's 97.19% Dr.
- On sensitive feature unlearning, Ferrari reduces feature sensitivity to 0.09 on CelebA mouth (vs 0.96 baseline) and MIA ASR to 51.28% (vs 84.36% baseline).
- On CelebA biased-mouth unlearning, Ferrari reaches 94.62% test accuracy, versus 95.87% baseline and 87.35% for the unlearn baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Minimize feature sensitivity with Ferrari | Ferrari · Sensitive feature unlearning (CelebA, Adult, Diabetes, IMDB) | feature sensitivity: 0.96 ±1.41×10−2 -> 0.09 ±3.04×10−4; 1.31 ±1.53×10−2 -> 0.05 ±1.72×10−4; 1.52 ±0.91×10−2 -> 0.07 ±1.07×10−4; 0.85 ±1.07×1 | Helped |
| Minimize feature sensitivity with Ferrari | Ferrari · Sensitive feature unlearning (CelebA, Adult, Diabetes, IMDB) | MIA attack success rate (%): 84.36 ±3.22 -> 51.28 ±2.41; 87.54 ±13.89 -> 49.58 ±1.38; 92.31 ±7.55 -> 42.61 ±1.81; 90.28 ±2.49 -> 43.75 ±1.86 | Helped |
| Minimize feature sensitivity with Ferrari | Ferrari · Sensitive feature unlearning (Table 1 datasets) | test accuracy (%): 79.46 ±2.32 -> 92.26 ±1.73; 65.27 ±0.58 -> 81.02 ±0.58; 64.19 ±0.72 -> 79.53 ±0.79; 83.27 ±2.05 -> 89.15 ±1.32 | Helped |
| Minimize feature sensitivity with Ferrari | Ferrari · Backdoor unlearning (MNIST, FMNIST, CIFAR-10, CIFAR-20, CIFAR-100, ImageNet) | accuracy on unlearn client dataset Du (%): 97.43 ±3.69 -> 0.11 ±0.01; 94.51 ±6.29 -> 0.90 ±0.03; 95.05 ±2.30 -> 0.29 ±0.04; 94.21 ±4.11 -> 0.78 ±0.08; 88.98 ±6.63 | Helped |
| Minimize feature sensitivity with Ferrari | Ferrari · Backdoor unlearning (MNIST, FMNIST, CIFAR-10, CIFAR-20, CIFAR-100, ImageNet) | accuracy on retain dataset Dr (%): 95.65 ±1.39 -> 95.93 ±0.45; 91.07 ±0.54 -> 92.83 ±0.61; 87.63 ±1.16 -> 89.91 ±0.95; 75.06 ±6.41 -> 78.29 ±3.12; 54.14 ±3 | Mixed |
| Minimize feature sensitivity with Ferrari | Ferrari · Biased feature unlearning (CMNIST Color; CelebA Mouth) | accuracy on retain dataset Dr (%): 64.94 ±7.88 -> 84.31 ±2.63; 79.46 ±2.09 -> 94.18 ±3.08 | Helped |
| Minimize feature sensitivity with Ferrari | Ferrari · Biased feature unlearning (CMNIST Color; CelebA Mouth) | accuracy on unlearn client dataset Du (%): 98.88 ±4.90 -> 84.62 ±3.59; 96.38 ±3.87 -> 94.79 ±1.48 | Hurt |
| Remove the denominator from the Ferrari feature-sensitivity loss (Non-Lipschitz variant) (ablation) | Ferrari · Backdoor unlearning | accuracy on retain dataset Dr (%): Dr accuracy drops below 10% | Hurt |
| Vary Gaussian noise level σ from 0.05 to 1.0 (ablation) | Ferrari · Backdoor feature unlearning | accuracy on Dr and Du | No clear effect |
| Use 70% rather than the full 100% of Du (ablation) | Ferrari · Backdoor feature unlearning | accuracy on Dr and Du: 70%; 100% | No clear effect |
| Use Non-IID client data (ablation) | Ferrari · CIFAR-10 backdoor unlearning | accuracy on Du: approximately 0.2% | No clear effect |
| Use Non-IID client data (ablation) | Ferrari · CIFAR-10 backdoor unlearning | accuracy on Dr: about 2% | Hurt |
Surprising findings
- Removing the Lipschitz denominator (Non-Lipschitz loss) causes catastrophic forgetting, with Dr accuracy dropping below 10%.
- Retraining with Gaussian noise on the unlearned feature region degrades accuracy substantially (e.g., over 27% drop), motivating the paper's alternative evaluation.
Limitations
- Performance degrades when only a small portion of the unlearn client's data is available (70% of Du comparable, less is worse).
- Evaluated only on classification models; generative models not covered.
- Theoretical analysis relies on assumptions such as zero training loss and an assumed feature removal threshold C.
Experimental setup
Datasets: MNIST Colored-MNIST Fashion-MNIST CIFAR-10 CIFAR-20 CIFAR-100 ImageNet CelebA Adult Diabetes IMDB
Benchmarks: CelebA Adult Diabetes IMDB MNIST Fashion-MNIST CIFAR-10 CIFAR-20 CIFAR-100 ImageNet CMNIST
Metrics: test accuracy retain accuracy Dr unlearn accuracy Du feature sensitivity attack success rate of model inversion runtime FLOPs
Baselines: Retrain Fine-tune FedCDP FedRecovery
Builds on: Lipschitz continuity FedRecovery FedCDP BadNets
Closest papers by meaning
- Federated Unlearning with Gradient Descent and Conflict Mitigation (AAAI 2025 · similarity 0.87)
- NoT: Federated Unlearning via Weight Negation (CVPR 2025 · similarity 0.87)
- Tackling Feature-Classifier Mismatch in Federated Learning via Prompt-Driven Feature Transformation (NeurIPS 2025 · similarity 0.86)
- Unlearning through Knowledge Overwriting: Reversible Federated Unlearning via Selective Sparse Adapter (CVPR 2025 · similarity 0.86)
- A Simple Data Augmentation for Feature Distribution Skewed Federated Learning (CVPR 2025 · similarity 0.85)
- Machine Unlearning in Low-Dimensional Feature Subspace (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.