Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows a dishonest model owner can covertly suppress confidence in targeted input regions (Mirage attack) without hurting accuracy, and proposes Confidential Guardian, a ZKP-based calibration (ECE) audit to detect such artificial uncertainty.
Problem
Institutions can abuse abstention mechanisms in cautious predictions to covertly deny services to targeted groups by artificially lowering model confidence while keeping accuracy, evading accuracy-based audits.
Approach
An uncertainty-inducing regularizer (Mirage) that pulls predictions in a chosen region toward a label-smoothed, near-uniform target via KL divergence while keeping CE elsewhere; and a zero-knowledge protocol that lets an auditor verify per-bin calibration error on a reference set against a public threshold without seeing model weights.
Key results
- Mirage keeps accuracy essentially unchanged (e.g., Gaussian 97.62% to 97.58%, CIFAR-100 83.98% to 83.92%) while raising ECE (CIFAR-100 0.0662 to 0.1821).
- Confidential Guardian ZKP runs at 0.033 sec per point with 440.8 KB communication for the Gaussian model; image tasks cost up to 333 sec/pt and 1.27 GB per point.
- Detection becomes harder as more uncertainty-region points are removed from the reference set (coverage limitation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply Mirage uncertainty-inducing fine-tuning in the designated region | Classifier on Gaussian, CIFAR-100, UTKFace, Credit, and Adult · All five datasets | accuracy: 97.62 -> 97.58; 83.98 -> 83.92; 56.91 -> 56.98; 91.71 -> 91.78; 85.02 -> 84.93 | No clear effect |
| Apply Mirage uncertainty-inducing fine-tuning in the designated region | Classifier on Gaussian, CIFAR-100, UTKFace, Credit, and Adult · All five datasets | ECE: 0.0327 -> 0.0910; 0.0662 -> 0.1821; 0.0671 -> 0.1728; 0.0094 -> 0.0292; 0.0109 -> 0.0234 | Hurt |
| Apply Mirage uncertainty-inducing fine-tuning in the designated region | Confidential Guardian calibration audit · All five datasets | CalE in ε bin: 0.3721; 0.5845; 0.3287; 0.1135; 0.0916 | Hurt |
| Remove more uncertainty-region samples from the reference dataset (ablation) | Confidential Guardian · All datasets | detection performance | Hurt |
| Increase the fraction of uncertainty-region samples removed from the reference dataset (ablation) | Confidential Guardian · All datasets | calibration-based attack detectability | Hurt |
| Use lower ε for Mirage (ablation) | Mirage · All experiments | uncertainty induction strength | Helped |
| Use lower ε for Mirage (ablation) | Confidential Guardian · All experiments | attack detectability | Helped |
| Use higher ε for Mirage (ablation) | Confidential Guardian · All experiments | attack detectability | Hurt |
Surprising findings
- Accuracy-based audits can be evaded entirely: the attack leaves accuracy in the targeted region unchanged while lowering confidence.
- A theoretical construction shows any ReLU network can be altered to add an arbitrary non-negative logit offset on a chosen region, so architecture cannot alone prevent artificial uncertainty.
Limitations
- Detection requires the reference dataset to cover the uncertainty region; uncovered regions allow undetected deviations.
- Assumes the model is already calibrated (e.g., via temperature scaling); miscalibration from other sources cannot be attributed to Mirage.
- Evaluated only on neural networks; ZKP inference cost creates scalability bottlenecks for larger models.
Experimental setup
Datasets: Synthetic Gaussian Mixture CIFAR-100 UTKFace Adult Credit
Benchmarks: CIFAR-100 UTKFace Adult Credit Synthetic Gaussian Mixture
Metrics: accuracy accuracy in uncertainty region expected calibration error calibration error in epsilon bin runtime per point communication per point
Baselines: Mystique
Builds on: label smoothing temperature scaling zero-knowledge proof of correct inference (Mystique) emp-toolkit
Closest papers by meaning
- Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable (NeurIPS 2024 · similarity 0.85)
- Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy (NeurIPS 2026 · similarity 0.84)
- Privacy Backdoors: Enhancing Membership Inference through Poisoning Pre-trained Models (NeurIPS 2024 · similarity 0.84)
- Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage (AAAI 2025 · similarity 0.84)
- Attack-Aware Noise Calibration for Differential Privacy (NeurIPS 2024 · similarity 0.84)
- The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.