Research map / Adversarial robustness and security / Attacks on text and graphs
Purify Unlearnable Examples via Rate-Constrained Variational Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D-VAE, a rate-constrained VAE with learnable class-wise embeddings that disentangles and removes unlearnable-example perturbations in a two-stage pre-training purification pipeline.
Problem
Unlearnable examples (availability poisoning with bounded perturbations) make models generalize poorly; existing defenses either are computationally expensive (adversarial training) or lack universality across UE types (JPEG/grayscale squeezing) or need extra clean data (diffusion purification).
Approach
Rate-constrained VAEs (KL-target floor) preferentially suppress high-predictivity, low-class-conditional-entropy perturbations; a D-VAE adds a learnable class-wise embedding and auxiliary decoder to estimate and subtract the perturbation, applied in two stages with different KL targets.
Key results
- On unlearnable CIFAR-10 (l_inf 8/255 and others), the method reaches 90.01% mean clean test accuracy versus 74.68% for adversarial training and 84.67% for AVATAR, close to 94.57% on clean data.
- On the 100-class ImageNet subset with four UE methods, mean accuracy is 75.00%, versus 57.59% for AT and 64.21% for ISS, with clean accuracy 80.52%.
- Mean performance drop on CIFAR-10 is 4.56%, compared with 9.90% for AVATAR and 11.93% for ISS, although the method fails to disentangle adversarial poisoning (TAP, SEP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply D-VAE two-stage purification instead of the strongest reported baseline | ResNet-18 trained on unlearnable CIFAR-10 · CIFAR-10 (mean across UEs) | accuracy: 84.67 -> 90.01 | Helped |
| apply D-VAE two-stage purification instead of ISS | ResNet-18 trained on unlearnable CIFAR-100 · CIFAR-100 (mean across UEs) | accuracy: 57.03 -> 66.20 | Helped |
| apply D-VAE two-stage purification instead of ISS | ResNet-18 trained on unlearnable 100-class ImageNet-subset · 100-class ImageNet-subset (mean across UEs) | accuracy: 64.21 -> 75.00 | Helped |
| remove the first purification stage (ablation) | D-VAE two-stage purification · CIFAR-10 (mean across UEs) | accuracy: 90.01 -> 80.52 | Hurt |
| remove inference 2 in the second stage (ablation) | D-VAE two-stage purification · CIFAR-10 (mean across UEs) | accuracy: 90.01 -> 89.02 | Hurt |
| remove the second purification stage (ablation) | D-VAE two-stage purification · CIFAR-10 (mean across UEs) | accuracy: 90.01 -> 36.21 | Hurt |
| remove inference 3 (ablation) | D-VAE two-stage purification · CIFAR-10 (mean across UEs) | accuracy: 90.01 -> 40.12 | Hurt |
| use the first-stage D-VAE for final reconstruction instead of the full two-stage procedure (ablation) | D-VAE purification · CIFAR-10 (mean across UEs) | accuracy: 90.01 -> 83.44 | Hurt |
Surprising findings
- Reducing KL divergence in a rate-constrained VAE removes UE perturbations more effectively than JPEG at matched PSNR.
- Perturbations estimated by D-VAE from clean samples can themselves act as poisoning, enabling detection of unlearnable datasets.
Limitations
- Fails to disentangle adversarial poisoning such as TAP and SEP, which the authors attribute to lower class-conditional entropy assumptions not holding.
- Hyperparameters (KLD targets) must be chosen empirically per dataset and encoder, using PSNR ranges as guidance.
- Evaluated only on supervised image classification with a limited set of UE attacks and models.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-subset (100 classes)
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-subset
Metrics: clean test accuracy PSNR MSE detection accuracy precision recall F1-score
Baselines: adversarial training adversarial augmentation image shortcut squeezing AVATAR LFU
Builds on: variational autoencoder image shortcut squeezing adversarial training
Closest papers by meaning
- Improving VAEs' Robustness to Adversarial Attack (ICLR 2021 · similarity 0.88)
- Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise (AAAI 2024 · similarity 0.88)
- Image Shortcut Squeezing: Countering Perturbative Availability Poisons with Compression (ICML 2023 · similarity 0.87)
- A3: Few-shot Prompt Learning of Unlearnable Examples with Cross-Modal Adversarial Feature Alignment (CVPR 2025 · similarity 0.86)
- Why do Variational Autoencoders Really Promote Disentanglement? (ICML 2024 · similarity 0.86)
- Detecting and Corrupting Convolution-based Unlearnable Examples (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.