Research map / Federated learning and privacy / Differential privacy
Neural Collapse meets Differential Privacy: Curious behaviors of NoisyGD with Near-Perfect Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models last-layer private fine-tuning via neural collapse (layer-peeled model) and proves that NoisyGD's misclassification error is dimension-independent when the feature shift parameter beta is below about p^{-1/2}, with PCA and feature centering proposed to restore robustness.
Problem
Explaining why differentially private fine-tuning of pre-trained models works despite high feature dimensionality, and how robust this dimension independence is to feature perturbations.
Approach
Use the neural collapse structure of last-layer features (ETF class means) with a feature shift parameter beta (l_inf distance to ideal features) to bound the 0-1 misclassification error of NoisyGD directly, giving log(1/gamma) sample complexity that is dimension-free for small beta; then analyze perturbations and propose PCA and mean-centering fixes.
Key results
- Empirical feature shift beta is about 0.10 median for ViT vs about 0.20 for ResNet-50 on CIFAR-10, with ViT giving smaller shift.
- Theory: NoisyGD sample complexity O(log(1/gamma)) dimension-free when beta <= p^{-1/2}; adversarial perturbations tighten the threshold to p^{-1}.
- Applying PCA to last-layer features (keeping K-1=10 components) improves NoisySGD test accuracy under perturbation while NoisySGD accuracy otherwise drops as dimension grows.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ImageNet-pretrained ViT instead of ImageNet-pretrained ResNet-50 for feature extraction | CIFAR-10 feature representations · CIFAR-10 training features | median feature shift β: β≈0.20 -> β≈0.10 | Helped |
| Add Gaussian perturbations to test features | Linear SGD · CIFAR-10 | accuracy | Hurt |
| Increase feature dimension while test features are perturbed | Linear SGD · CIFAR-10 | accuracy | No clear effect |
| Increase feature dimension while test features are perturbed | NoisySGD · CIFAR-10 | accuracy | Hurt |
| Add Gaussian perturbations to test features | NoisySGD · CIFAR-10 | accuracy | Hurt |
| Project perturbed training and test features onto the top 10 PCA components (ablation) | NoisySGD · CIFAR-10 | accuracy | Helped |
| Project perturbed training and test features onto the top 100 PCA components (ablation) | NoisySGD · CIFAR-10 | robustness | No clear effect |
| Combine class imbalance with a private feature offset of ||ν||∞ =0.1 | NoisyGD · Synthetic perfect neural collapse | utility | Hurt |
| Perturb test data with ||ν||∞=0.1 | NoisyGD · Synthetic perfect neural collapse | utility | Hurt |
Surprising findings
- Adversarial perturbations lower the tolerable beta threshold from p^{-1/2} to p^{-1}, making them more harmful than random perturbations.
- Under DP, NoisyGD becomes non-robust to small offset perturbations even though non-private GD is unaffected.
Limitations
- Theory focuses on one-step or projected NoisyGD on linear last-layer models with idealized ETF structure.
- Experiments limited to CIFAR-10 with a single fine-tuning setup; NC may not consistently occur in other transfer tasks.
- Feature normalization is not robust to class imbalance.
Experimental setup
Datasets: CIFAR-10 ImageNet
Benchmarks: CIFAR-10
Metrics: test accuracy feature shift parameter beta sample complexity bound
Baselines: Linear SGD (non-private) NoisySGD
Builds on: Neural Collapse NoisySGD zCDP De et al. 2022 DP pre-training
Closest papers by meaning
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (NeurIPS 2025 · similarity 0.86)
- Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.86)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.86)
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime (ICML 2025 · similarity 0.85)
- Controlling Neural Collapse Enhances Out-of-Distribution Detection and Transfer Learning (ICML 2025 · similarity 0.85)
- Navigate Beyond Shortcuts: Debiased Learning Through the Lens of Neural Collapse (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.