Research map / Learning theory and algorithms
Prediction and decision losses: research map
2,134 accepted papers on Prediction and decision losses in Learning theory and algorithms, from ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), grouped into 6 clusters and 23 approaches. The busiest year so far is 2026.
Within Learning theory and algorithms, its share held steady from 24.0% in 2023–24 to 23.7% in 2025–26 (535 → 576 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).
Explore Prediction and decision losses in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
Approaches and key papers
selection · robust · test · 1,009 papers
Approaches in this cluster:
- Active learning and subset selection (199 papers)
Select informative samples or subsets with optimization-based and importance-weighted algorithms. - Adaptive testing and experiments (201 papers)
Sequential decision and testing procedures for validity, treatment effects and instance selection. - Distributionally robust optimization (184 papers)
Train models against worst-case distributions using robust optimization formulations. - Data attribution and recourse (160 papers)
Influence functions, robust recourse and decision-oriented learning under shifts. - Theory of neural network sensitivity (143 papers)
Analyze learnability and robustness of networks through linear structure and Lipschitz control. - Determinantal point process inference (122 papers)
Scalable MAP inference and learning for determinantal point processes and related online methods.
Most cited and most cited since 2024:
- Designing Network Design Spaces (CVPR 2020 · 1,958 citations)
- CatBoost: unbiased boosting with categorical features (NeurIPS 2018 · 1,125 citations)
- Variable Importance in High-Dimensional Settings Requires Grouping (AAAI 2024 · 14 citations)
- The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution (AAAI 2024 · 14 citations)
ranking · nearest · neighbor · 262 papers
Approaches in this cluster:
- Ranking models from comparisons (127 papers)
Model and infer rankings and preferences from pairwise or partial comparisons, e.g. Mallows models. - Approximate nearest neighbor indexing (58 papers)
Build graph and partition indexes for fast approximate nearest neighbor search with guarantees. - Metric and ordinal embedding (47 papers)
Learn distance-preserving embeddings from metric or ordinal information. - Locality-sensitive hashing (30 papers)
Hash data to binary codes or buckets for sublinear similarity search.
Most cited and most cited since 2024:
- To Trust Or Not To Trust A Classifier (NeurIPS 2018 · 278 citations)
- Representation Tradeoffs for Hyperbolic Embeddings (ICML 2018 · 190 citations)
- The Linear Representation Hypothesis and the Geometry of Large Language Models (ICML 2024 · 10 citations)
- Active preference learning for ordering items in- and out-of-sample (NeurIPS 2024 · 5 citations)
losses · loss functions · surrogate · 251 papers
Approaches in this cluster:
- Consistency of surrogate losses (72 papers)
Prove calibration and consistency bounds for convex surrogates of classification losses. - Losses for decision-making (68 papers)
Design losses for learning to defer, predict-then-optimize and risk criteria beyond average error. - Loss landscape analysis (61 papers)
Characterize local minima and critical points in neural network loss surfaces. - Ranking-based loss optimization (50 papers)
Optimize rank-based and multivariate losses with consistent or differentiable surrogates.
Most cited and most cited since 2024:
- Learning Deep Embeddings with Histogram Loss (NeurIPS 2016 · 257 citations)
- Cross-Entropy Loss Functions: Theoretical Analysis and Applications (ICML 2023 · 217 citations)
- Model Alignment as Prospect Theoretic Optimization (ICML 2024 · 23 citations)
- Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize (AAAI 2024 · 11 citations)
conformal · predictions · online · 233 papers
Approaches in this cluster:
- Conformal prediction sets (116 papers)
Construct prediction sets with coverage guarantees and improved efficiency or conditional validity. - Algorithms with explicit predictors (62 papers)
Model prediction error and predictor quality to analyze algorithms using ML predictions. - Probabilistic calibration (55 papers)
Calibrate classifier and regression uncertainty with proper scores and calibration guarantees.
Most cited and most cited since 2024:
- Verified Uncertainty Calibration (NeurIPS 2019 · 138 citations)
- Conformalized Quantile Regression (NeurIPS 2019 · 49 citations)
- Conformal Risk Control (ICLR 2024 · 26 citations)
- Conformal Prediction Sets Improve Human Decision Making (ICML 2024 · 8 citations)
explanations · shapley · attribution · 201 papers
Approaches in this cluster:
- Provable post-hoc explanations (92 papers)
Formalize and certify fidelity, robustness and sensitivity of black-box model explanations. - Shapley value attribution (63 papers)
Estimate Shapley values efficiently for feature attribution and data valuation. - Counterfactual explanations and recourse (46 papers)
Generate actionable counterfactuals and recourse for decisions of trained models.
Most cited and most cited since 2024:
- Axiomatic Attribution for Deep Networks (ICML 2017 · 2,911 citations)
- Understanding Black-box Predictions via Influence Functions (ICML 2017 · 1,328 citations)
- Shaping Up SHAP: Enhancing Stability through Layer-Wise Neighbor Selection (AAAI 2024 · 18 citations)
- Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles (AAAI 2024 · 16 citations)
trees · decision tree · forests · 178 papers
Approaches in this cluster:
- Tree ensemble optimization (77 papers)
Improve, prune and interpret random forests and boosted tree ensembles. - Optimal decision tree learning (75 papers)
Compute optimal or sparse decision trees with algorithmic and generalization guarantees. - Compact interpretable model classes (26 papers)
Learn small interpretable models such as diagrams, lattices and oblique trees.
Most cited and most cited since 2024:
- LightGBM: A Highly Efficient Gradient Boosting Decision Tree (NeurIPS 2017 · 9,433 citations)
- Neural Additive Models: Interpretable Machine Learning with Neural Nets (NeurIPS 2021 · 121 citations)
- Symbolic Regression Enhanced Decision Trees for Classification Tasks (AAAI 2024 · 15 citations)
- Generative Model for Decision Trees (AAAI 2024 · 5 citations)
Related topics in Learning theory and algorithms
- Matrix and tensor methods (1,074)
- Sample complexity (1,363)
- Combinatorial and search optimization (1,492)
- Fairness (516)
- Submodular and game algorithms (1,030)
- Kernels and regression theory (1,004)
- Clustering (725)
- Optimal transport (384)
