Research map / Label-efficient and robust learning / Recognition and long-tail
Group-robust Sample Reweighting for Subpopulation Shifts via Influence Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GSR, a two-stage method that learns representations on group-unlabeled data, then iteratively reweights held-out samples via influence functions (implicit differentiation) while retraining the last layer, using a small group-labeled target set.
Problem
Improving worst-group accuracy under subpopulation shift with limited group labels, by using group-labeled data to guide sample weights of group-unlabeled data rather than training on it directly.
Approach
Derives exact sample-weight gradients via implicit differentiation (influence functions) for the bilevel minimax objective, made tractable and strongly convex by last-layer retraining with L2 regularization, and adaptively aggregates group influence scores; the Hessian term distinguishes it from the one-step MAPLE approximation.
Key results
- GSR achieves 82.5 average worst-group accuracy across four datasets, about 1.0 point above DFR (81.5), using the same group-label budget.
- GSR reaches 78.5% worst-group accuracy on MultiNLI, versus 74.7% for DFR.
- The Hessian-free variant GSR-HF averages 79.4, showing the Hessian term matters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GSR instead of DFR | group-robust training · CivilComments | worst-group accuracy (%): 70.1_{\pm0.8} -> 71.7_{\pm 0.6} | Helped |
| use GSR instead of DFR | group-robust training · Waterbirds | worst-group accuracy (%): 92.9_{\pm0.2} -> 92.9_{\pm 0.0} | No clear effect |
| use GSR instead of DFR | group-robust training · CelebA | worst-group accuracy (%): 88.3_{\pm1.1} -> 87.0_{\pm 0.4} | Hurt |
| use GSR instead of DFR | group-robust training · MultiNLI | worst-group accuracy (%): 74.7_{\pm0.7} -> 78.5_{\pm 0.3} | Helped |
| use GSR instead of DFR | group-robust training · average across four datasets | worst-group accuracy (%): 1.0% | Helped |
| add the Hessian to sample-weight updates (ablation) | GSR · Waterbirds | worst-group accuracy (%): 87.5_{\pm 0.1} -> 92.9_{\pm 0.0} | Helped |
| add the Hessian to sample-weight updates (ablation) | GSR · CelebA | worst-group accuracy (%): 86.3_{\pm 0.4} -> 87.0_{\pm 0.4} | Helped |
| add the Hessian to sample-weight updates (ablation) | GSR · MultiNLI | worst-group accuracy (%): 74.7_{\pm 0.0} -> 78.5_{\pm 0.3} | Helped |
| add the Hessian to sample-weight updates (ablation) | GSR · CivilComments | worst-group accuracy (%): 68.9_{\pm 0.2} -> 71.7_{\pm 0.6} | Helped |
| retrain the full network with the exact Hessian instead of using last-layer retraining (ablation) | GSR · ColoredMNIST-S | worst-group accuracy (%): 91.2 -> 94.1 | Helped |
| approximate the Hessian with LiSSA during full-network retraining (ablation) | GSR full-parameter retraining · ColoredMNIST-S | worst-group accuracy (%): 94.1 -> 80.2 | Hurt |
| use last-layer retraining instead of full-network retraining with the exact Hessian (ablation) | GSR · ColoredMNIST-S | worst-group accuracy (%): 94.1 -> 91.2 | Hurt |
Surprising findings
- Reweighting with GSR is robust to up to 40% label corruption in the held-out set, with corrupted minority samples receiving near-zero weights.
- Better worst-group models are not trained with equal group weights, and worst-group gains come at the cost of mean accuracy.
Limitations
- Mean accuracy decreases relative to some baselines due to the worst-group trade-off.
- Only retrains the last layer; representation learning is not improved, and full-network retraining with exact Hessian is computationally prohibitive.
- Relies on the assumption that ERM representations are sufficiently good.
Experimental setup
Datasets: Waterbirds CelebA MultiNLI CivilComments ColoredMNIST
Benchmarks: Waterbirds CelebA MultiNLI CivilComments
Metrics: worst-group accuracy mean accuracy
Baselines: Group DRO RWG JTT CnC SSA MAPLE DFR SELF
Builds on: MAPLE DFR influence functions Just Train Twice
Closest papers by meaning
- Optimal Transport Reweighting for Robust Learning under Spurious Correlations and Label Noise (NeurIPS 2026 · similarity 0.86)
- Enhancing Robustness of Last Layer Two-Stage Fair Model Corrections (NeurIPS 2024 · similarity 0.86)
- Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference (ICML 2024 · similarity 0.86)
- Diverse Prototypical Ensembles Improve Robustness to Subpopulation Shift (ICML 2025 · similarity 0.86)
- Unsupervised Group Re-identification via Adaptive Clustering-Driven Progressive Learning (AAAI 2024 · similarity 0.85)
- The Group Robustness is in the Details: Revisiting Finetuning under Spurious Correlations (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.