Research map / Label-efficient and robust learning / Recognition and long-tail
DPA: A one-stop metric to measure bias amplification in classification datasets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Directional Predictability Amplification (DPA), a bounded, directional, predictability-based metric that measures relative change in attacker predictability to quantify bias amplification in both balanced and unbalanced classification datasets.
Problem
Existing bias amplification metrics either fail on balanced datasets, cannot capture negative amplification, or (for predictability-based leakage amplification) cannot identify the direction of amplification; they are also sensitive to attacker choice and unbounded.
Approach
Measure dataset and model bias as predictability of attribute from task (and vice versa) via attacker functions, then define amplification as the normalized relative change (M minus D over M plus D) in each direction, which is directional, bounded in [-1,1], and less sensitive to attacker hyperparameters.
Key results
- On balanced ImSitu A→T, DPA's model ranking exactly matches the Sen_A→T ranking, while LA and Multi_→ show large mismatches.
- On the unbalanced COCO masking experiment, only DPA and BA_→ increase with the attribution score across masking versions; on balanced COCO, only DPA captures the rising T→A model bias (0.023 to 0.104).
- In the COMPAS balanced set, BA_→ reports zero amplification while DPA reports 0.061 (T→A) and 0.100 (A→T).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPA instead of BA→ to measure T→A amplification in balanced COCO as masking increases | ViT-b-16 on COCO · COCO balanced, original to bounding-box masked | bias amplification T→A: 0.0230_{0.0008} -> 0.1039_{0.0006}; BA→: 0.0000_{0.0000} -> 0.0000_{0.0000} | Helped |
| Progressively mask the person | ViT-b-16 on COCO · COCO unbalanced, original to bounding-box masked | bias amplification T→A: DPA: -0.0152_{0.0017} -> 0.8085_{0.0003}; BA→: -0.0227_{0.0001} -> 0.0601_{0.0002} | Mixed |
| Measure balanced hidden bias with DPA rather than BA→ | VGG16 on balanced COCO with one-pixel shortcuts · Balanced COCO hidden-bias experiment | bias amplification A→T: DPA: 0.0829_{0.0007} -> 0.1380_{0.0010} | Helped |
| Use DPA rather than BA→ on balanced COMPAS | Decision tree on COMPAS · COMPAS balanced | bias amplification: BA→: 0.000_{0.000}; DPA: 0.061_{0.008} (T→A), 0.100_{0.004} (A→T) | Helped |
| Use Multi→ rather than DPA to measure COMPAS amplification | Decision tree on COMPAS · COMPAS unbalanced, A→T | bias amplification A→T: DPA: -0.004_{0.002}; Multi→: 0.038_{0.001} | Hurt |
| Use relative rather than absolute change in predictability (ablation) | DPA attacker-function calculation · Synthetic dataset and ImSitu | attacker-score stability | Helped |
| Use DPA rather than LA for relative change in predictability | Bias amplification measurement · Across balanced and unbalanced classification datasets | reliability | Helped |
Surprising findings
- Co-occurrence metric BA_→ reports zero bias amplification on balanced datasets by construction, even when hidden biases from unlabeled objects are amplified by models.
- On the unbalanced COCO masking experiment, LA and BA_MALS report scores that do not track the increasing model bias.
Limitations
- DPA aggregates across all attribute and task values and cannot give per-A-T-pair bias amplification.
- Metric choice depends on the bias notion; BA_→ may be preferred for some datasets such as hiring data, and DPA may under-report amplification in those cases.
- Predictability scores depend on the quality function; cross-entropy losses must be transformed to fit the bounded formulation.
Experimental setup
Datasets: COCO ImSitu COMPAS
Benchmarks: COCO masking experiment ImSitu balanced and unbalanced sub-datasets COMPAS one-pixel shortcut controlled experiment
Metrics: bias amplification score rank agreement with sensitivity score confidence intervals
Baselines: BA_MALS BA_→ Multi_→ LA (leakage amplification)
Builds on: leakage amplification (Wang et al.) BA_→ directional bias amplification Multi_→ Men Also Like Shopping (BA_MALS)
Closest papers by meaning
- An Effective Theory of Bias Amplification (ICLR 2025 · similarity 0.86)
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.84)
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers (CVPR 2025 · similarity 0.84)
- Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing (NeurIPS 2025 · similarity 0.84)
- Evaluating Model Bias Requires Characterizing its Mistakes (ICML 2024 · similarity 0.84)
- Mitigating Spurious Correlations via Disagreement Probability (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.