Research map / Adversarial robustness and security / Attacks on text and graphs
Data Poisoning Attacks against Conformal Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a black-box data poisoning attack framework that manipulates conformal prediction set sizes (uncertainty) rather than misclassifications, using closed-form influence-based updates and worst-case model optimization.
Problem
Data poisoning attacks against conformal prediction (CP) uncertainty estimates are unexplored; existing poisoning attacks target misclassification, and existing CP attacks focus on validity rather than efficiency (set size).
Approach
A bi-level poisoning framework that crafts poisons to inflate or shrink CP set sizes for target samples, approximating the discrete set sizes and non-differentiable quantile with surrogate losses, optimizing against the worst-case poisoned model via sharpness-aware perturbation, and using closed-form first- and second-order influence updates to avoid retraining.
Key results
- On CIFAR-10 with HPS at a 2% poison budget, the attack achieves a 0.48 set size reduction ratio versus below 0.32 for random-noise baselines.
- Under MaxUp, adversarial poisoning, and EPIC defenses, the attack retains set size reduction of 0.34-0.42 for HPS/APS/RAPS/RSCP at 2% budget (e.g., 0.34 with HPS under MaxUp vs 0.46 with no defense).
- Second-order update reaches 0.48 reduction at 2% budget with ~369 min, versus first-order 0.46 at ~19 min and MetaPoison 0.38 at ~495 min.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed poisoning attack instead of random uniform or Gaussian noise | Conformal prediction attacks · CIFAR-10, HPS and RAPS, 2% poison budget | set size reduction ratio: 0.48 with HPS and 0.54 with RAPS; baselines below 0.32 | Helped |
| Use the proposed poisoning attack under existing poisoning defenses | Overconfidence CP attacks on CIFAR-10 · CIFAR-10, 2% poison budget, HPS | set size reduction ratio: 0.46 ± 0.02 -> 0.34 ± 0.05 | Mixed |
| Use second-order rather than first-order optimization (ablation) | Proposed poisoning attack · CIFAR-10, 2% poison budget | set size reduction ratio: 0.46 ± 0.02 -> 0.48 ± 0.02 | Helped |
| Use first-order rather than second-order optimization (ablation) | Proposed poisoning attack · CIFAR-10, 2% poison budget | running time (min): 369.23 ± 4.81 -> 19.26 ± 0.14 | Helped |
| Use worst-case ℓ3 loss rather than ℓ2 loss (ablation) | Proposed poisoning attack · CIFAR-10, 0.5% and 1% poison budgets | set size reduction ratio: +0.01 and +0.02 | Helped |
| Increase the perturbation bound (ablation) | Overconfidence CP attacks · CIFAR-10, 2% poison budget | set size reduction ratio | Helped |
Surprising findings
- Existing poisoning defenses (MaxUp, adversarial poisoning, EPIC) remain largely ineffective against the uncertainty-targeting attack, with only modest drops in set size reduction.
Limitations
- Experiments limited to image classification on three datasets and small-to-medium CNN architectures.
- Full CP attack is only sketched in the main text; details deferred to a full version.
- Attack relies on an auxiliary dataset and a surrogate model trained comparably to the victim.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet
Metrics: set size reduction ratio set size expansion ratio prediction consistency empirical coverage rate running time
Baselines: RandUn RandGa MetaPoison
Builds on: conformal prediction MetaPoison sharpness-aware minimization influence functions
Closest papers by meaning
- Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning (ICLR 2025 · similarity 0.88)
- Robust Yet Efficient Conformal Prediction Sets (ICML 2024 · similarity 0.88)
- Verifiably Robust Conformal Prediction (NeurIPS 2024 · similarity 0.88)
- The Pitfalls and Promise of Conformal Inference Under Adversarial Attacks (ICML 2024 · similarity 0.87)
- Sharpness-Aware Data Poisoning Attack (ICLR 2024 · similarity 0.85)
- Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.