Research map / Adversarial robustness and security / Certified and empirical robustness
The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows conformal prediction fails on non-robust models under l_inf PGD attacks and that adversarial training variants (TRADES, MART, FAT) inflate prediction set size, then proposes AT-UR (entropy minimization plus Beta-weighting by true class rank) to reduce PSS while keeping robustness.
Problem
Conformal prediction under standard adversarial attacks yields uninformative prediction sets for non-robust models, and improved adversarial training methods increase prediction set size, making CP inefficient.
Approach
Adds an entropy minimization term and a Beta-density importance weight on the adversarial cross-entropy loss, indexed by the normalized true-class probability rank (TCPR), to train adversarially robust models with smaller conformal prediction sets; a theorem bounds expected PSS by the Beta-weighted loss at the population level.
Key results
- Under AutoAttack on Caltech256, AT-Beta reduces PSS from 23.73 (AT) to 18.54 on the same dataset while keeping coverage near 90%; on CIFAR100 AT-Beta gives PSS 11.61 vs 14.29 for AT.
- Under PGD100 on Caltech256, AT-Beta gives PSS 35.39 vs 43.20 for AT at roughly 90% coverage.
- Non-robust models yield PSS close to the number of classes under PGD100 attacks, versus 1.03 and 2.39 on clean CIFAR10/CIFAR100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Beta weighting by TCPR to adversarial training (ablation) | AT, FAT, and TRADES · CIFAR10, CIFAR100, Caltech256, and CUB200 | PSS: AT: 2.54(0.04) -> 2.50(0.04), 14.29(0.59) -> 11.61(0.40), 23.73(1.68) -> 18.54(1.32), 17.75(0.71) -> 16.56(0.76); FAT: 2 | Mixed |
| Add entropy minimization to adversarial training (ablation) | AT, FAT, and TRADES · CIFAR10, CIFAR100, Caltech256, and CUB200 | PSS: AT: 2.54(0.04) -> 2.45(0.04), 14.29(0.59) -> 13.29(0.49), 23.73(1.68) -> 21.19(1.46), 17.75(0.71) -> 16.47(0.61); FAT: 2 | Mixed |
| Add both Beta weighting and entropy minimization (ablation) | AT, FAT, and TRADES · CIFAR10, CIFAR100, Caltech256, and CUB200 | PSS: AT: 2.54(0.04) -> 2.50(0.04), 14.29(0.59) -> 11.78(0.47), 23.73(1.68) -> 18.56(1.32), 17.75(0.71) -> 16.67(0.58); FAT: 2 | Mixed |
| Use TRADES or MART instead of standard AT | Adversarially trained ResNet50 with APS · CIFAR10, CIFAR100, Caltech256, and CUB200 | PSS: TRADES vs AT: 2.49(0.03) vs 2.54(0.04), 12.22(0.50) vs 14.29(0.59), 22.82(1.37) vs 23.73(1.68), 22.29(0.96) vs 17.75(0.7 | Mixed |
| Use standard training instead of adversarial training | APS, RAPS, and RSCP · CIFAR10 and CIFAR100 under strong adversarial attacks | PSS | Hurt |
| Use focal loss weighting of hard samples instead of Beta weighting of promising TCPR samples (ablation) | AT-Focal · CIFAR100 and Caltech256 | PSS: CIFAR100: 23.79 -> 27.24; Caltech256: 43.20 -> 48.35 | Hurt |
| Add label smoothing (ablation) | AT · Caltech256 | PSS: 43.20 -> 46.39 | Hurt |
| Vary the Beta distribution parameter b across the tested values (ablation) | AT-Beta · Caltech256 | PSS: (1.1, 2.0): 37.09(2.24); (1.1, 3.0): 35.22(2.76); (1.1, 4.0): 35.55(2.28); (1.1, 5.0): 35.39(2.66); (1.1, 6.0): 38.54(3. | Mixed |
Surprising findings
- TRADES and MART, which improve top-1 robust accuracy, substantially increase prediction set size relative to standard AT.
- Focal-style weighting emphasizing hard samples worsens CP efficiency (Caltech256 PSS 48.35 vs 43.20 for AT), while weighting promising samples helps.
Experimental setup
Datasets: CIFAR10 CIFAR100 Caltech-256 CUB200
Benchmarks: CIFAR10 CIFAR100 Caltech-256 CUB200
Metrics: coverage prediction set size top-1 clean accuracy top-1 robust accuracy normalized PSS
Baselines: AT TRADES MART FAT APS RAPS RSCP Conformal training (uncertainty-aware AT) focal loss label smoothing
Builds on: adversarial training (PGD) TRADES MART Fair-AT adaptive prediction sets randomly smoothed conformal prediction
Closest papers by meaning
- Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability (ICML 2025 · similarity 0.91)
- Verifiably Robust Conformal Prediction (NeurIPS 2024 · similarity 0.90)
- Robust Yet Efficient Conformal Prediction Sets (ICML 2024 · similarity 0.89)
- Provably Robust Conformal Prediction with Improved Efficiency (ICLR 2024 · similarity 0.88)
- Efficient Robust Conformal Prediction via Lipschitz-Bounded Networks (ICML 2025 · similarity 0.88)
- Data Poisoning Attacks against Conformal Prediction (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.