Research map / Adversarial robustness and security / Attacks on text and graphs
CausalDiff: Causality-Inspired Disentanglement via Diffusion Model for Adversarial Defense
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CausalDiff, a causality-inspired conditional diffusion model that disentangles label-causative and non-causative latent factors via a causal information bottleneck objective, then purifies and classifies adversarial inputs for robustness against unseen attacks.
Problem
Neural classifiers remain vulnerable to adversarial attacks, especially unseen attack types; existing adversarial training, certified defenses, and purification generalize poorly to unforeseen attacks.
Approach
Model data generation with a structural causal model in which label-causative factors S and label-non-causative factors Z jointly generate the input via a conditional diffusion model; train with a causal information bottleneck objective (mutual information terms plus CLUB disentanglement penalty) and at inference purify the input with the unconditional diffusion model, infer S and Z by likelihood
Key results
- CausalDiff reaches 86.39% average robust accuracy on CIFAR-10 across StAdv, AutoAttack l_inf and l_2 (+4.01% over prior SOTA), with 90.23% clean accuracy.
- On CIFAR-100, CausalDiff achieves 56.25% robust accuracy (+3.13%), though with lower clean accuracy (65.62%).
- On GTSRB, CausalDiff achieves 82.62% average robust accuracy (+4.93%), with 97.85% clean accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CausalDiff instead of the strongest reported baseline | CIFAR-10 defense · CIFAR-10 unseen attacks (average) | robust accuracy (%): 82.38 -> 86.39 | Helped |
| Use CausalDiff instead of DiffPure | CIFAR-10 defense · CIFAR-10 BPDA + EOT, ℓ∞ | robust accuracy (%): 81.40 -> 88.48 | Helped |
| Use CausalDiff instead of RDC | CIFAR-100 defense · CIFAR-100 AutoAttack, ℓ∞ | robust accuracy (%): 53.12 -> 56.25 | Helped |
| Use CausalDiff instead of AT-4 | GTSRB defense · GTSRB AutoAttack and fog (average) | robust accuracy (%): 77.69 -> 82.62 | Helped |
| Add causal factor inference to the AP-only variant (ablation) | CausalDiff · CIFAR-10 unseen attacks (average) | robust accuracy (%): 77.48 -> 86.39 | Helped |
| Add adversarial purification to the variant without AP (ablation) | CausalDiff · CIFAR-10 unseen attacks (average) | robust accuracy (%): 81.77 -> 86.39 | Helped |
| Add the CLUB disentanglement term (η = 10⁻⁵ rather than η = 0) (ablation) | CausalDiff training · CIFAR-10 AutoAttack, ℓ∞ | robust accuracy (%): 65.04% -> 69.14% | Helped |
| Use encoder-predicted latent variables instead of diffusion-based causal factor inference (ablation) | CausalDiff inference · CIFAR-10 AutoAttack, ℓ∞ | robust accuracy (%): 0.00% | Hurt |
| Disentangle label-causative and non-causative factors (ablation) | Toy causal model · Toy data, 100-step PGD, ε = 6.4 | robust accuracy (%): at least 71.8% vs. 61.4% | Helped |
| Add disentanglement to the causal toy model (ablation) | Toy causal model · Toy data | prediction margin (ℓ₂): 9.58 -> 10.64 | Helped |
| Add disentanglement to the causal toy model (ablation) | Toy causal model · Toy data | Δp(y|v): 0.32 -> 0.22 | Helped |
Surprising findings
- CausalDiff's clean accuracy on CIFAR-100 (65.62%) is much lower than the undefended WRN40-2 baseline (78.13%), which the authors attribute to limited samples and a weaker DDPM backbone.
- Disentanglement alone (CausalDiff w/o AP) outperforms all baselines except RDC on average robustness, suggesting that modeling in-domain generation improves inherent robustness.
Limitations
- Clean accuracy drops notably on CIFAR-100 relative to standard classifiers, attributed to limited training data and a weaker backbone.
- Larger disentanglement weight eta causes the diffusion model to collapse, limiting tuning range.
- Built on DDPM rather than the stronger EDM backbone, so reported results may be understated.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 GTSRB EDM-generated images (CIFAR-100 augmentation)
Benchmarks: CIFAR-10 CIFAR-100 GTSRB
Metrics: clean accuracy robust accuracy average robust accuracy
Baselines: AT-l_inf AT-l_2 CausalAdv DICE DiffPure LM-DDPM LM-EDM SBGC CAMA RDC ADP GDMP
Builds on: DDPM Likelihood Maximization purification CLUB mutual information upper bound DiffAE CausalAdv
Closest papers by meaning
- Disentangled Representation Learning in Non-Markovian Causal Systems (NeurIPS 2024 · similarity 0.85)
- AdvAD: Exploring Non-Parametric Diffusion for Imperceptible Adversarial Attacks (NeurIPS 2024 · similarity 0.85)
- Diffusion Models Demand Contrastive Guidance for Adversarial Purification to Advance (ICML 2024 · similarity 0.85)
- Certified Causal Defense with Generalizable Robustness (AAAI 2025 · similarity 0.85)
- Can Diffusion Models Disentangle? A Theoretical Perspective (NeurIPS 2025 · similarity 0.85)
- DiffAdvMAP: Flexible Diffusion-Based Framework for Generating Natural Unrestricted Adversarial Examples (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.