Research map / Learning theory and algorithms / Fairness
Robust ML Auditing using Prior Knowledge
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a prior-based framework for manipulation-proof ML fairness auditing, showing public priors let platforms fool auditors and deriving detection-rate bounds for a labeled dataset prior, validated on ACSEmployment and CelebA.
Problem
Fairness audits of black-box ML systems are vulnerable to audit manipulation (fairwashing), where a platform alters its answers to auditors while behaving differently for other users.
Approach
Formalizes the auditor's prior as a set of expectable models and uses a labeled dataset held privately by the auditor to define a ball-shaped prior; derives the probability of detecting an optimal projection-based manipulation under demographic parity, showing public priors enable manipulation.
Key results
- Theorem gives closed-form detection probability P_uf for a ball dataset prior under uniform prior assumption; if tau equals delta, P_uf = 1, and if delta = 0, P_uf = 0.
- Across datasets the platform can conceal roughly 10 to 20 points of demographic parity gap, with manipulation strategies (Linear Relaxation and ROC) being most effective.
- Concealable unfairness converges to zero with increasing audit budget only for the CelebA Smiling target, due to low aleatoric uncertainty; in other settings some unfairness remains undetectable even at budgets up to 5,000.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a public auditor prior known to the platform | Fairness audit · Black-box fairness auditing | manipulation detection | Hurt |
| Use a private labeled-dataset prior | Robust ML auditing · Black-box fairness auditing | manipulation detection | Helped |
| Increase the audit budget | Dataset-prior audit with LeNet on CelebA Smiling · CelebA Smiling | concealable unfairness | Helped |
| Increase the audit budget | Dataset-prior audit on other evaluated settings · CelebA targets other than Smiling and ACSEmployment | concealable unfairness | No clear effect |
| Use Linear Relaxation or ROC Mitigation for manipulation | Platform manipulation during fairness audits · CelebA and ACSEmployment | concealable unfairness | Helped |
| Use fairness-repair manipulation strategies | LeNet, GBDT, and Logistic Regression · CelebA and ACSEmployment | concealable unfairness: from $10$ to $20$ points | Helped |
Surprising findings
- If the auditor's prior is public, the platform can always pick a model in the intersection of the prior and the fair set and appear both fair and honest.
- Even with high audit budgets, a platform can retain capacity to conceal unfairness in most configurations, giving a negative answer to RQ2.
Limitations
- Analysis assumes uninformative (uniform) prior over models in the auditor prior, and a worst-case batch auditing setting where all queries are known to the platform.
- Threshold tau is hard to calibrate; the paper leaves systematic calibration to future work.
- Results focus on demographic parity and binary classifiers, with a one-time audit that does not cover post-audit model changes.
Experimental setup
Datasets: ACSEmployment (folktables, Minnesota 2018) CelebA
Benchmarks: ACSEmployment CelebA
Metrics: demographic parity gap (concealable unfairness) detection score detection rate
Baselines: ROC OT-L LinR ThreshOpt
Builds on: fairwashing via explanations demographic parity fairness repair methods (ROC, Linear Relaxation, optimal transport, threshold op
Closest papers by meaning
- Auditing Sabotage Bench: A Benchmark for Detecting and Fixing Research Sabotage in ML Codebases (NeurIPS 2026 · similarity 0.86)
- Adaptive auditing of AI systems with anytime-valid guarantees (NeurIPS 2026 · similarity 0.86)
- Auditing Local Explanations is Hard (NeurIPS 2024 · similarity 0.85)
- How Well Can Differential Privacy Be Audited in One Run? (NeurIPS 2025 · similarity 0.84)
- Fraud-Proof Revenue Division on Subscription Platforms (ICML 2025 · similarity 0.83)
- Sequentially Auditing Differential Privacy (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.