Research map / Adversarial robustness and security / Certified and empirical robustness
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OPSA, a label-free attack that maximizes a soft conformal set size, and OPSA-AT, an adversarial conformal training defense that reduces prediction set size while keeping coverage on CIFAR-10/100 and mini-ImageNet.
Problem
Adversarial attacks inflate conformal prediction set sizes and degrade efficiency; existing defenses are attack-specific or sacrifice set compactness, so a defense that keeps sets small and coverage valid under varied attacks is needed.
Approach
An attack (OPSA) that maximizes a sigmoid-based soft set size using the perturbed true-class logit as the threshold, so it needs no significance level; a defense (OPSA-AT) trains with OPSA perturbations on a bi-objective loss combining a soft classification loss and soft size loss, with the threshold computed by exact batch-wise quantile calibration.
Key results
- On CIFAR-10 under PGD10 attack, OPSA-AT achieves size 3.24 at ~89.3% coverage versus 4.46 for PGD10 adversarial training and 6.63 for BETA-AT.
- On CIFAR-100 clean inputs, OPSA-AT yields set size 8.41 versus 73.65 for BETA-AT, with coverage 89.38%.
- OPSA attack produces the largest prediction set sizes against most non-OPSA-trained defenses, e.g. size 7.29 vs 7.06 for PGD-based training on CIFAR-10 FGSM-trained model under the OPSA attack.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with OPSA-AT rather than TRADES^10 | ResNet34 · CIFAR-10, OPSA^10 attack | Size: 4.02 ± 0.02 -> 3.11 ± 0.03 | Helped |
| Train with OPSA-AT rather than TRADES^10 | ResNet34 · CIFAR-100, OPSA^10 attack | Size: 32.30 ± 0.19 -> 32.07 ± 0.25 | Helped |
| Train with OPSA-AT rather than MART^10 | ResNet50 · mini-ImageNet, OPSA^10 attack | Size: 30.84 ± 0.19 -> 27.55 ± 0.21 | Helped |
| Use OPSA rather than baseline attacks | CIFAR-10 defense models · CIFAR-10 | Size | Helped |
| Use OPSA rather than baseline attacks | CIFAR-100 defense models · CIFAR-100 | Size | Helped |
| Use OPSA-AT rather than baseline defenses | ResNet34 · CIFAR-100, clean | Accuracy: 52.10% -> 55.65% | Helped |
| Use OPSA-AT rather than baseline defenses | ResNet50 · mini-ImageNet, clean | Accuracy: 45.13% -> 55.65% | Helped |
| Increase attack temperature T1 from 0.001 to 0.1 (ablation) | OPSA · CIFAR-100, OPSA attack | Size: 22.06 ± 0.20 -> 29.35 ± 0.26 | Helped |
| Increase attack temperature T1 from 0.1 to 1 (ablation) | OPSA · CIFAR-100, OPSA attack | Size: 29.35 ± 0.26 -> 33.30 ± 0.27 | Helped |
| Increase attack temperature T1 from 1 to 10 (ablation) | OPSA · CIFAR-100, OPSA attack | Size: 33.30 ± 0.27 -> 33.50 ± 0.25 | Helped |
| Increase attack temperature T1 from 10 to 1000 (ablation) | OPSA · CIFAR-100, OPSA attack | Size: 33.50 ± 0.26 -> 33.50 ± 0.26 | No clear effect |
Surprising findings
- BETA-AT yields very large set sizes (e.g., 73.65 on CIFAR-100 clean) despite its adversarial training, indicating poor efficiency.
- Reported OPSA attack does not consistently yield the lowest accuracy, yet produces the largest set sizes.
Limitations
- Experiments limited to ResNet architectures and small image datasets; the authors plan PreAct ResNet experiments.
- Batch sizes (64 and 200) are smaller than the 500-1000 the authors theorize is ideal for exchangeability, due to hardware limits.
- Coverage on evaluation does not hold exactly at 1-alpha in all cases; OPSA-AT is slower to train than most baselines.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 mini-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 mini-ImageNet
Metrics: coverage set size SSCV accuracy
Baselines: FGSM PGD TRADES MART BETA-AT Square APGD AutoAttack
Builds on: Threshold Response (THR) conformal prediction conformal training TRADES MART BETA-AT
Closest papers by meaning
- The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks (ICML 2024 · similarity 0.91)
- Provably Cost-Sensitive Adversarial Defense via Randomized Smoothing (ICML 2025 · similarity 0.87)
- Towards Million-Scale Adversarial Robustness Evaluation With Stronger Individual Attacks (CVPR 2025 · similarity 0.87)
- PROSAC: Provably Safe Certification for Machine Learning Models under Adversarial Attacks (AAAI 2025 · similarity 0.87)
- Efficient Robustness Evaluation via Constraint Relaxation (AAAI 2025 · similarity 0.87)
- On the Scalability of Certified Adversarial Robustness with Generated Data (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.