Research map / Learning theory and algorithms / Prediction and decision losses
Embracing Contradiction: Theoretical Inconsistency Will Not Impede the Road of Building Responsible AI Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A position paper arguing that theoretical inconsistency among Responsible AI metrics (fairness definitions, accuracy-privacy trade-offs) should be embraced rather than eliminated, and proposing documentation practices for inconsistency.
Problem
Responsible AI metrics such as fairness definitions, privacy, and neutrality are often mathematically incompatible, and the field typically treats these conflicts as flaws to be fixed by choosing a single metric or enforcing consistency.
Approach
Treat metric inconsistency as a feature: keep a full suite of potentially contradictory metrics as divergent objectives, justified by normative pluralism, epistemological completeness, and implicit regularization, and document normative provenance and inconsistency levels (magnitude, directionality, locality, sensitivity) rather than enforcing consistency.
Key results
- Reports that Bell et al. found plenty of approximately fair models across 18 real-world datasets, suggesting impossibility results often overstate practical trade-offs.
- Cites Fisher et al. evaluating 9 LLMs on 7,314 political queries, where refusal achieved 100% safety scores but scored poorly on utility.
- Cites an example where PCGrad improves NYUv2 mean IoU by 2.3 points while reducing depth RMSE by 0.04m.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PCGrad during semantic-depth co-training | PCGrad · NYUv2 semantic-depth co-training | mean IoU: +2.3 percentage points | Helped |
| Apply PCGrad during semantic-depth co-training | PCGrad · NYUv2 semantic-depth co-training | depth RMSE: reducing depth RMSE by 0.04m | Helped |
| Add a differential-privacy loss | Medical imaging model · External-hospital test | AUC: 5% | Helped |
| Add a differential-privacy loss | Medical imaging model · Internal test | AUC: 2% decline | Hurt |
| Use refusal techniques to approximate political neutrality | Political neutrality techniques · Political queries | safety: 100% | Helped |
Surprising findings
- Argues that impossibility results often overstate practical trade-offs, since loosening zero disparity to minimum disparity yields many approximately fair models.
- Claims that a differential-privacy loss in medical imaging improved external-hospital AUC by 5% despite a 2% internal decline.
Limitations
- No new experiments; the argument relies on cited empirical results from other papers.
- Multi-objective optimization adds computational overhead and possible training instability, as acknowledged by the authors.
- Definitions and theorems are informal, and the proposed protocol (provenance sheets, SPHERE cards) is not validated empirically.
Experimental setup
Builds on: Kleinberg impossibility theorem Chouldechova impossibility result PCGrad CAGrad Model Cards
Closest papers by meaning
- Rigor in AI: Doing Rigorous AI Work Requires a Broader, Responsible AI-Informed Conception of Rigor (NeurIPS 2025 · similarity 0.88)
- Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Duties (AAAI 2024 · similarity 0.85)
- Trustworthy AI Must Account for Interactions (NeurIPS 2026 · similarity 0.84)
- Towards a Theory of AI Personhood (AAAI 2025 · similarity 0.84)
- Position: A Roadmap to Pluralistic Alignment (ICML 2024 · similarity 0.84)
- Utility Engineering: Analyzing and Controlling Emergent Value Systems in AIs (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.