Research map / Learning theory and algorithms / Prediction and decision losses
Data attribution and recourse
Influence functions, robust recourse and decision-oriented learning under shifts.
160 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Prediction and decision losses, keyword cluster “selection · robust · test”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
54 papers in 2023–24 and 45 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 10.1% to 7.8%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Satisfying Real-world Goals with Dataset Constraints (NeurIPS 2016 · 122 citations)
- 2017 · Analyzing Computer Vision Data - The Good, the Bad and the Ugly (CVPR 2017 · 25 citations)
- 2018 · Human-in-the-Loop Interpretability Prior (NeurIPS 2018 · 44 citations)
- 2019 · On the Accuracy of Influence Functions for Measuring Group Effects (NeurIPS 2019 · 82 citations)
- 2020 · Regression under Human Assistance (AAAI 2020 · 37 citations)
- 2021 · Adaptive Machine Unlearning (NeurIPS 2021 · 43 citations)
- 2022 · Deep Unlearning via Randomized Conditionally Independent Hessians (CVPR 2022 · 60 citations)
- 2023 · Predictive Multiplicity in Probabilistic Classification (AAAI 2023 · 23 citations)
- 2024 · Variable Importance in High-Dimensional Settings Requires Grouping (AAAI 2024 · 14 citations)
- 2025 · Learning About Algorithm Auditing in Five Steps: Scaffolding How High School Youth Can Systematically and Critically Evaluate Machine Learning Applications (AAAI 2025 · 10 citations)
- 2026 · Robust Strategic Classification under Decision-Dependent Cost Uncertainty (ICML 2026 · 0 citations)
Earliest papers here
- Satisfying Real-world Goals with Dataset Constraints (NeurIPS 2016 · 122 citations)
- Logistic Boosting Regression for Label Distribution Learning (CVPR 2016 · 75 citations)
- Deep Learning for Algorithm Portfolios (AAAI 2016 · 68 citations)
What has been tried
136 papers here report 1,184 measured changes (a component added, removed or swapped, compared with a baseline): helped 63%, no clear effect 11%, hurt 19%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add internal stacking | Synthetic grouping benchmark · Type-I error | Helped | Variable Importance in High-Dimensional Settings Requires Grouping (AAAI 2024 · 14 citations) | |
| Refine the forget set by memorization and apply Fine-tune sequentially to homogeneous subsets instead of once to the full set | CIFAR-10 · ToW | 0.849±0.030 -> 0.937±0.052 | Helped | What makes unlearning hard and what to do about it (NeurIPS 2024 · 10 citations) |
| Optimize λ for variance rather than fixing λ = 1 | ImageNet validation · mean-squared error | No clear effect | AutoEval Done Right: Using Synthetic Data for Model Evaluation (ICML 2025 · 1 citation) | |
| fine-tune for 1 epoch and increase the pruning ratio | CIFAR-10 · forget accuracy drop | 8.7% -> 3.7% | Hurt | DisCEdit: Model Editing by Identifying Discriminative Components (NeurIPS 2024 · 1 citation) |
| Replace Center MAE with Center MSE | Linear and nonlinear datasets · success probability | .67±.01 vs .67±.01; .63±.19 vs .63±.19; .95±.02 vs .94±.05; .66±.36 vs .64±.36; .26±.09 vs .15±.11; .40±.16 vs .37±.18; | Mixed | Gradient-Based Nonlinear Rehearsal Learning with Multivariate Alterations (AAAI 2025 · 0 citations) |
| use feature-only rather than feature-label signal planting | Vehicle evaluation, signal planting · test-time success | Hurt | Statistical Collusion by Collectives on Learning Platforms (ICML 2025 · 0 citations) | |
| add the proposed second-order loss-change term | CIFAR-10N Worst mislabeled detection · ROC AUC | 41.52 -> 95.93 | Helped | Towards Robust Influence Functions with Flat Validation Minima (ICML 2025 · 0 citations) |
| Add uncertainty-aware 95% quantile decisions instead of deterministic 50% quantile decisions | electricity · AUC of combined cost | 2.5884 ± 0.13 -> 2.6056 ± 0.14 | Hurt | When to retrain a machine learning model (ICML 2025 · 0 citations) |
| Add average CVaR loss alongside cross entropy | CIFAR-10, IPC 10 · accuracy | 46.7 -> 49.1 | Helped | Group Distributionally Robust Dataset Distillation with Risk Minimization (ICLR 2025 · 0 citations) |
| Use amortized estimator instead of per-point likelihood estimator (10 samples per point) | MiniBooNE, CIFAR-10, IMDB · Pearson correlation with ground truth | MiniBooNE 0.858→0.994; CIFAR-10 0.077→0.893; IMDB 0.342→0.930 | Helped | Scaling Laws for the Value of Individual Data Points in Machine Learning (ICML 2024 · 0 citations) |
| Use online-ensemble estimation instead of OGD with a fixed step size | Market-managing data · mean-square error | Mixed | Avoiding Undesired Future with Minimal Cost in Non-Stationary Environments (NeurIPS 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 33 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 6 of 33 papers
- MNIST · 4 of 33 papers
- SVHN · 2 of 33 papers
- IMDB · 2 of 33 papers
Baselines
- QWZ23 · 2 of 33 papers
- SAC · 2 of 33 papers
- Random · 2 of 33 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 99 papers; from OpenAlex and the papers' affiliations).
- Harvard University · 7 papers
- Stanford University · 7 papers
- Google · 6 papers
- Massachusetts Institute of Technology · 5 papers
- National University of Singapore · 4 papers
- Technion · 4 papers
- University of California, Berkeley · 4 papers
- University of Chicago · 3 papers
- Inria · 3 papers
- Nanjing University · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Active learning and subset selection (199 papers)
- Adaptive testing and experiments (201 papers)
- Distributionally robust optimization (184 papers)
- Theory of neural network sensitivity (143 papers)
- Determinantal point process inference (122 papers)
Elsewhere in the topic
- Ranking models from comparisons (127 papers)
- Conformal prediction sets (116 papers)
- Provable post-hoc explanations (92 papers)
- Tree ensemble optimization (77 papers)
- Optimal decision tree learning (75 papers)
- Consistency of surrogate losses (72 papers)
- Losses for decision-making (68 papers)
- Shapley value attribution (63 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).