Research map / Learning theory and algorithms / Fairness
Learning with Statistical Equality Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives a generalization theory for equality-constrained statistical learning and proposes a primal-dual (dual ascent) algorithm that solves sequences of unconstrained empirical Lagrangian problems, demonstrated on fairness, BVPs, and interpolating classifiers.
Problem
Constrained learning with statistical equality constraints (e.g., fairness, boundary value problems, interpolation) lacks approximation and generalization guarantees, since existing results only cover inequality constraints.
Approach
Extends strong duality and generalization bounds of constrained learning to equality constraints under additional regularity (interior/Slater-type and Lipschitz-in-parameters) assumptions, showing the bound trades off model capacity, sample size, and constraint sensitivity, and uses dual ascent on the empirical Lagrangian instead of penalty weight tuning.
Key results
- On the convection BVP with beta=30, the constrained approach achieves 0.62 ± 0.17% relative L2 error versus 2.46 ± 0.99% for PINN.
- On CIFAR-100 the constrained interpolating classifier reaches 75.17 ± 0.21% test accuracy versus 76.11 ± 0.28% for ERM, while CIFAR-10 is essentially equal (95.01% vs 95.03%).
- Equality formulation on COMPAS achieves the lowest group rate disparity, comparable to an inequality relaxation with tight tolerance (epsilon=1e-4), while epsilon=1e-2 introduces noticeable differences.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use equality constraints instead of fixed-multiplier PINN losses | Convection BVP · Convection BVP (β=30) | relative L² error: 2.46 ± 0.99 % -> 0.62 ± 0.17 % | Helped |
| Use equality constraints instead of fixed-multiplier PINN losses | Convection BVP · Convection BVP (β=50) | relative L² error: 56.0 ± 25.8 % -> 30.7 ± 24.2 % | Helped |
| Use classwise interpolation constraints instead of ERM | CIFAR-10 classifier · CIFAR-10 | test accuracy: 95.03 ± 0.21 % -> 95.01 ± 0.10 % | No clear effect |
| Use classwise interpolation constraints instead of ERM | CIFAR-100 classifier · CIFAR-100 | test accuracy: 76.11 ± 0.28 % -> 75.17 ± 0.21 % | Hurt |
| Relax exact demographic-parity equalities to double-sided inequalities with ε=10⁻⁴ (ablation) | COMPAS classifier · COMPAS | group disparity | No clear effect |
| Relax exact demographic-parity equalities to double-sided inequalities with ε=10⁻² (ablation) | COMPAS classifier · COMPAS | group disparity | Hurt |
| Set prescribed positive-outcome rates to rⱼ=0.5 | Prescribed-rate fairness model · COMPAS | test accuracy | No clear effect |
Surprising findings
- For classwise interpolation on CIFAR-100, the dual variables correlate strongly (0.89) with per-class test error, offering a label-free signal of class difficulty.
Limitations
- Equality constraints require stronger regularity assumptions (Lipschitz parametrization, interior of constraint epigraph) than inequality-constrained learning.
- Primal recovery is not guaranteed: the averaged Lagrangian iterates and last iterate may be infeasible or non-monotone.
- Results are for approximating with rich parametrizations; experiments are small-scale (COMPAS, CIFAR, a convection BVP) and the sigmoid relaxation of indicators is used in fairness experiments.
Experimental setup
Datasets: COMPAS CIFAR-10 CIFAR-100
Benchmarks: COMPAS CIFAR-10 CIFAR-100 convection BVP
Metrics: mean accuracy group rate disparity relative L2 error test accuracy
Baselines: unconstrained ERM inequality relaxation of equality constraints PINN with fixed weights DP-constrained learning
Builds on: constrained learning duality (non-convex, rich parametrizations) PINNs Slater's condition
Closest papers by meaning
- Fairness via Independence: A General Regularization Framework for Machine Learning (ICLR 2026 · similarity 0.89)
- Too Relaxed to Be Fair (ICML 2020 · similarity 0.88)
- Near-Optimal Solutions of Constrained Learning Problems (ICLR 2024 · similarity 0.88)
- How Far Can Fairness Constraints Help Recover From Biased Data? (ICML 2024 · similarity 0.86)
- Post-hoc bias scoring is optimal for fair classification (ICLR 2024 · similarity 0.86)
- Kernel-based Equalized Odds: A Quantification of Accuracy-Fairness Trade-off in Fair Representation Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.