Research map / Reinforcement learning / Policy gradient and actor-critic

A Distribution Mapping Approach to Counterfactually Fair Reinforcement Learning

NeurIPS 2026 · Reinforcement learning › Policy gradient and actor-critic · approach: Distributional value learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CFSMDM, a data-preprocessing method that estimates counterfactual states and rewards via sequential quantile (distribution) mapping so offline RL policies become counterfactually fair under weaker, non-additive noise assumptions, with suboptimality and unfairness bounds.

Problem

Counterfactually fair policy learning in sequential decision-making (RL) when sensitive attributes affect states and rewards, particularly where existing preprocessing (CFSDP) requires additive noise.

Approach

Sequentially match conditional quantile levels of observed states and rewards across sensitive-attribute values (sequential marginal distribution mapping), so counterfactual states/rewards are recovered under strict monotonicity in exogenous noise rather than additivity; then run standard offline RL (FQI) on the preprocessed data.

Key results

• On PowerED (education attribute), CFSMDM achieves unfairness 0.11 (SD 0.03) versus 0.14 for CFSDP and 0.45 for Full, at policy value 57.27 comparable to baselines.
• In simulations, CFSMDM's CF metric decreases as training sample size N grows, outperforming all non-random baselines on fairness.
• CFSMDM is slightly more unfair than CFSDP and Unaware for the sex attribute on PowerED (0.09 vs 0.07 and 0.06).

Surprising findings

• On PowerED, excluding the sensitive attribute ('Unaware') is already fairer than Full and nearly as fair as CFSDP for sex, and CFSMDM is slightly more unfair than Unaware for sex.

Limitations

• Finite-sample quantile estimation errors leave residual unfairness; CF metric may be inaccurate under quantile misspecification or assumption violations.
• Requires fitting multiple quantile models over a grid of levels, which is more computationally costly than CFSDP.
• Requires strict monotonicity of states/rewards in exogenous noise and an exchangeability/causal DAG assumption.

Experimental setup

Datasets: simulated CMDP with nonadditive noise PowerED study dataset

Benchmarks: simulated CMDP PowerED

Metrics: CF metric (counterfactual action disagreement rate) discounted cumulative reward (policy value)

Baselines: Full Unaware Random FLAP_M ECOCF_M CFSDP

Builds on: CFSDP FLAP fitted Q iteration quantile regression counterfactual fairness

Closest papers by meaning

• Counterfactual Realizability (ICLR 2025 · similarity 0.85)
• A Causal Lens for Learning Long-term Fair Policies (ICLR 2025 · similarity 0.85)
• Variational Counterfactual Intervention Planning to Achieve Target Outcomes (ICML 2025 · similarity 0.85)
• FairDICE: Fairness-Driven Offline Multi-Objective Reinforcement Learning (NeurIPS 2025 · similarity 0.84)
• Counterfactual Fairness by Combining Factual and Counterfactual Predictions (NeurIPS 2024 · similarity 0.84)
• Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding (ICLR 2024 · similarity 0.84)