Research map / Reinforcement learning / Policy gradient and actor-critic
Adapting Static Fairness to Sequential Decision-Making: Bias Mitigation Strategies towards Equal Long-term Benefit Rate
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ELBERT, a ratio-after-aggregation long-term fairness notion for MDPs, and ELBERT-PO, a PPO variant with a fairness-aware advantage that mitigates long-term group bias while keeping reward high.
Problem
Static fairness constraints applied to sequential decisions can worsen long-term bias, and existing long-term fairness notions (ratio-before-aggregation) cause temporal discrimination; no bias-mitigation method exists for ratio-after-aggregation long-term fairness because its policy gradient is not
Approach
Define long-term group well-being as cumulative group supply over cumulative group demand within an MDP (SD-MDP), and show the gradient of the squared bias reduces to a standard policy gradient with a fairness-aware advantage function, so PPO can be used directly; soft-max/min smoothing handles multiple groups.
Key results
- In lending, ELBERT-PO achieves bias 0.02, reducing G-PPO bias by 87.5% and R-PPO and A-PPO bias by over 75%, with high reward.
- In infectious disease control, ELBERT-PO achieves the lowest bias of 0.01 and matches G-PPO reward, while R-PPO reaches the same bias with much lower reward.
- In attention allocation (multi-group), ELBERT-PO achieves the lowest bias and the highest reward among all methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ELBERT-PO instead of G-PPO | Lending · Lending | bias: 87.5% | Helped |
| Use ELBERT-PO instead of R-PPO or A-PPO | Lending · Lending | bias: over $75\%$ | Helped |
| Use ELBERT-PO instead of R-PPO | Infectious disease control · Infectious disease control | reward | Helped |
| Use ELBERT-PO instead of G-PPO | Infectious disease control · Infectious disease control | reward | No clear effect |
| Use ELBERT-PO instead of A-PPO | Infectious disease control · Infectious disease control | bias | Helped |
| Use ELBERT-PO instead of the baselines | Attention allocation · Attention allocation | reward | Helped |
| Increase the bias coefficient $\alpha$ (ablation) | ELBERT-PO · Attention allocation | bias | Helped |
| Increase the bias coefficient $\alpha$ (ablation) | ELBERT-PO · Attention allocation | convergence speed | Hurt |
| Increase the bias coefficient $\alpha$ (ablation) | ELBERT-PO · Attention allocation | reward | Helped |
Surprising findings
- Larger bias coefficient alpha led to slightly higher rewards, suggesting lower bias need not cost reward.
- Directly injecting historical bias into immediate reward (R-PPO) can achieve low bias but substantially harms reward.
Limitations
- ELBERT only regularizes supply-demand ratios and not the demand level itself; extra demand constraints are needed in some applications.
- Soft bias with temperature beta is only an upper bound approximation of the max-min bias, with gap up to 2 log M / beta.
- Evaluated only on simulated environments rather than real-world data.
Experimental setup
Datasets: lending simulation environment infectious disease control simulation environment attention allocation simulation environment
Benchmarks: lending (credit approval) infectious disease control (harder version) attention allocation (harder version)
Metrics: average reward long-term bias (max-min benefit rate difference)
Baselines: A-PPO G-PPO R-PPO
Builds on: PPO constrained MDP A-PPO equal opportunity demographic parity
Closest papers by meaning
- A Causal Lens for Learning Long-term Fair Policies (ICLR 2025 · similarity 0.91)
- Remembering to Be Fair: Non-Markovian Fairness in Sequential Decision Making (ICML 2024 · similarity 0.91)
- Fair Participation via Sequential Policies (AAAI 2024 · similarity 0.88)
- Long-Term Fair Decision Making through Deep Generative Models (AAAI 2024 · similarity 0.87)
- Fairness Shields: Safeguarding against Biased Decision Makers (AAAI 2025 · similarity 0.87)
- Fair Off-Policy Learning from Observational Data (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.