Research map / Label-efficient and robust learning / Recognition and long-tail
Reproducibility study of FairAC
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reproduces the FairAC fair attribute completion framework on graphs, checks its five claims on NBA, Pokec-z/n, and extends it to new datasets, sensitive attributes, and individual fairness (consistency).
Problem
Whether the claims of FairAC (fair attribute completion for graphs with missing node attributes, addressing feature and topological unfairness) hold under independent reproduction, and whether it generalizes beyond the original datasets and sensitive attributes.
Approach
Independent reproduction of FairAC with a refactored, reusable codebase; the study tests the claims on the original datasets, adds German Credit and Recidivism, evaluates age and gender as sensitive attributes on Pokec-z, and adds a consistency metric for individual fairness.
Key results
- On Pokec-z, FairAC reduces statistical parity difference to 0.55 and equal opportunity difference to 0.13 versus 1.56 and 3.17 for FairGNN, with accuracy 65.33 vs 68.16 for FairGNN.
- With age as sensitive attribute on Pokec-z, FairAC's fairness degrades sharply (delta SP 27.46) and GCN outperforms both fair models, which the authors attribute to age being important for the prediction task.
- Consistency stays near-identical across GCN, FairGNN and FairAC (about 41.33 to 41.35 on Pokec-z), suggesting no trade-off between group and individual fairness in their experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FairAC instead of GCN | GCN downstream classifier · NBA | ΔSP+ΔEO: 0.71 \pm 0.18 -> 0.19 \pm 0.08 | Helped |
| Use FairAC instead of FairGNN | GCN downstream classifier · Pokec-z | ΔSP+ΔEO: 4.73 \pm 1.47 -> 0.68 \pm 0.09 | Helped |
| Use FairAC instead of FairGNN | GCN downstream classifier · Pokec-n | ΔSP+ΔEO: 0.85 \pm 0.31 -> 0.58 \pm 0.76 | Helped |
| Add adversarial training to BaseAC (ablation) | Attribute completion on Pokec-z with α=0.1 · Pokec-z, α=0.1 | ΔSP+ΔEO: 0.48 \pm 0.21 -> 0.76 \pm 0.90 | Hurt |
| Add adversarial training to BaseAC (ablation) | Attribute completion on Pokec-z with α=0.3 · Pokec-z, α=0.3 | ΔSP+ΔEO: 1.08 \pm 1.32 -> 0.68 \pm 0.09 | Helped |
| Add adversarial training to BaseAC (ablation) | Attribute completion on Pokec-z with α=0.5 · Pokec-z, α=0.5 | ΔSP+ΔEO: 0.47 \pm 0.77 -> 0.35 \pm 0.50 | Helped |
| Add adversarial training to BaseAC (ablation) | Attribute completion on Pokec-z with α=0.8 · Pokec-z, α=0.8 | ΔSP+ΔEO: 0.99 \pm 0.86 -> 0.27 \pm 0.43 | Helped |
| Vary the missing-attribute rate α from 0.1 to 0.8 (ablation) | FairAC · Pokec-z | ΔSP+ΔEO: 0.76 \pm 0.90; 0.68 \pm 0.09; 0.35 \pm 0.50; 0.27 \pm 0.43 | Mixed |
| Use age rather than region as the sensitive attribute (ablation) | FairAC on Pokec-z · Pokec-z | ΔSP+ΔEO: 0.68 \pm 0.09 -> 47.36 \pm 4.38 | Hurt |
| Use gender rather than region as the sensitive attribute (ablation) | FairAC on Pokec-z · Pokec-z | ΔSP+ΔEO: 0.68 \pm 0.09 -> 0.96 \pm 0.52 | Mixed |
| Increase β to emphasize the sensitive classifier (ablation) | FairAC · Pokec-z | group fairness | Helped |
Surprising findings
- Age as sensitive attribute on Pokec-z causes a drastic fairness drop, with the plain GCN outperforming both fair models.
- At beta=0.8 the reproduction shows a marginal fairness increase, contrary to the original study's continuous decline.
Limitations
- FairAC was trained on only 25% of each dataset due to memory constraints of the original machines.
- The generic-framework claim could only be partially tested given the breadth of the claim.
- Some original details (hyperparameters, auto-encoder pretraining, seeds) were only found in the codebase or after contacting the authors.
Experimental setup
Datasets: NBA Pokec-z Pokec-n German Credit Recidivism
Benchmarks: NBA Pokec-z Pokec-n German Credit Recidivism
Metrics: accuracy AUC statistical parity difference equal opportunity difference consistency
Baselines: GCN FairGNN ALFR ALFR-e Debias Debias-e FCGE BaseAC
Builds on: FairAC FairGNN DeepWalk PyGDebias
Closest papers by meaning
- Reproducibility Study Of Learning Fair Graph Representations Via Automated Data Augmentations (NeurIPS 2024 · similarity 0.76)
- Statistical Inference for Fairness Auditing (NeurIPS 2024 · similarity 0.76)
- The Unreasonable Effectiveness of Open Science in AI: A Replication Study (AAAI 2025 · similarity 0.76)
- Position: Let’s Strengthen Verifiability if We Can’t Enforce Reproducibility (NeurIPS 2026 · similarity 0.75)
- RAGChecker: A Fine-grained Framework for Diagnosing Retrieval-Augmented Generation (NeurIPS 2024 · similarity 0.74)
- Parallel Empirical Evaluations: Resilience despite Concurrency (AAAI 2024 · similarity 0.74)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.