Research map / Learning theory and algorithms
Prediction and decision losses: research map
2,400 papers on Prediction and decision losses in Learning theory and algorithms, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 6 clusters and 23 approaches. The busiest year so far is 2025.
What it is
How models are trained and judged: which surrogate losses lead to the right decisions, explaining predictions (for example with Shapley values), decision trees and interpretable models, ranking and nearest-neighbour search, calibration and robust selection of data.
Terms you will see on this page:
- Surrogate loss
- an easy-to-optimise loss used in place of the true objective, such as 0-1 error.
- Shapley value
- a game-theory way to split a prediction among the input features.
- Calibration
- predicted probabilities matching how often events actually happen.
Start reading with the most cited papers here:
- LightGBM: A Highly Efficient Gradient Boosting Decision Tree (NeurIPS 2017 · 9,433 citations)
- Axiomatic Attribution for Deep Networks (ICML 2017 · 2,911 citations)
Within Learning theory and algorithms, its share held steady from 24.0% in 2023–24 to 23.7% in 2025–26 (535 → 576 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Prediction and decision losses in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Conformal prediction sets rising: 3% → 16% of the topic (16 → 90 papers)
- Shapley value attribution rising: 3% → 5% of the topic (18 → 27 papers)
- Approximate nearest neighbor indexing rising: 2% → 3% of the topic (9 → 16 papers)
- Theory of neural network sensitivity falling: 9% → 5% of the topic (48 → 27 papers)
- Active learning and subset selection falling: 11% → 7% of the topic (57 → 40 papers)
Papers from 2025–2026 that are already cited most:
- HotSpot: Signed Distance Function Optimization with an Asymptotically Sufficient Condition (CVPR 2025 · 10 citations)
- Learning About Algorithm Auditing in Five Steps: Scaffolding How High School Youth Can Systematically and Critically Evaluate Machine Learning Applications (AAAI 2025 · 10 citations)
- Conformal Thresholded Intervals for Efficient Regression (AAAI 2025 · 6 citations)
- Finsler Multi-Dimensional Scaling: Manifold Learning for Asymmetric Dimensionality Reduction and Embedding (CVPR 2025 · 3 citations)
Approaches and key papers
Data selection and robustness · 1,097 papers
Keywords: selection, robust, test
Approaches in this cluster:
- Active learning and subset selection (214 papers)
Select informative samples or subsets with optimization-based and importance-weighted algorithms. - Adaptive testing and experiments (215 papers)
Sequential decision and testing procedures for validity, treatment effects and instance selection. - Distributionally robust optimization (215 papers)
Train models against worst-case distributions using robust optimization formulations. - Data attribution and recourse (180 papers)
Influence functions, robust recourse and decision-oriented learning under shifts. - Theory of neural network sensitivity (145 papers)
Analyze learnability and robustness of networks through linear structure and Lipschitz control. - Determinantal point process inference (128 papers)
Scalable MAP inference and learning for determinantal point processes and related online methods.
Most cited and most cited since 2024:
- Designing Network Design Spaces (CVPR 2020 · 1,958 citations)
- CatBoost: unbiased boosting with categorical features (NeurIPS 2018 · 1,125 citations)
- Variable Importance in High-Dimensional Settings Requires Grouping (AAAI 2024 · 14 citations)
- The Evidence Contraction Issue in Deep Evidential Regression: Discussion and Solution (AAAI 2024 · 14 citations)
Loss functions · 317 papers
Keywords: losses, loss functions, surrogate
Approaches in this cluster:
- Consistency of surrogate losses (98 papers)
Prove calibration and consistency bounds for convex surrogates of classification losses. - Losses for decision-making (74 papers)
Design losses for learning to defer, predict-then-optimize and risk criteria beyond average error. - Loss landscape analysis (71 papers)
Characterize local minima and critical points in neural network loss surfaces. - Ranking-based loss optimization (74 papers)
Optimize rank-based and multivariate losses with consistent or differentiable surrogates.
Most cited and most cited since 2024:
- Learning Deep Embeddings with Histogram Loss (NeurIPS 2016 · 257 citations)
- Cross-Entropy Loss Functions: Theoretical Analysis and Applications (ICML 2023 · 217 citations)
- Model Alignment as Prospect Theoretic Optimization (ICML 2024 · 23 citations)
- Leaving the Nest: Going beyond Local Loss Functions for Predict-Then-Optimize (AAAI 2024 · 11 citations)
Ranking and nearest neighbours · 297 papers
Keywords: ranking, nearest, neighbor
Approaches in this cluster:
- Ranking models from comparisons (135 papers)
Model and infer rankings and preferences from pairwise or partial comparisons, e.g. Mallows models. - Approximate nearest neighbor indexing (71 papers)
Build graph and partition indexes for fast approximate nearest neighbor search with guarantees. - Metric and ordinal embedding (56 papers)
Learn distance-preserving embeddings from metric or ordinal information. - Locality-sensitive hashing (35 papers)
Hash data to binary codes or buckets for sublinear similarity search.
Most cited and most cited since 2024:
- To Trust Or Not To Trust A Classifier (NeurIPS 2018 · 278 citations)
- Representation Tradeoffs for Hyperbolic Embeddings (ICML 2018 · 190 citations)
- The Linear Representation Hypothesis and the Geometry of Large Language Models (ICML 2024 · 10 citations)
- Active preference learning for ordering items in- and out-of-sample (NeurIPS 2024 · 5 citations)
Conformal prediction and calibration · 265 papers
Keywords: conformal, predictions, online
Approaches in this cluster:
- Conformal prediction sets (132 papers)
Construct prediction sets with coverage guarantees and improved efficiency or conditional validity. - Algorithms with explicit predictors (67 papers)
Model prediction error and predictor quality to analyze algorithms using ML predictions. - Probabilistic calibration (66 papers)
Calibrate classifier and regression uncertainty with proper scores and calibration guarantees.
Most cited and most cited since 2024:
- Verified Uncertainty Calibration (NeurIPS 2019 · 138 citations)
- Conformalized Quantile Regression (NeurIPS 2019 · 49 citations)
- Conformal Risk Control (ICLR 2024 · 26 citations)
- Conformal Prediction Sets Improve Human Decision Making (ICML 2024 · 8 citations)
Explanations and attribution · 223 papers
Keywords: explanations, shapley, attribution
Approaches in this cluster:
- Provable post-hoc explanations (106 papers)
Formalize and certify fidelity, robustness and sensitivity of black-box model explanations. - Shapley value attribution (69 papers)
Estimate Shapley values efficiently for feature attribution and data valuation. - Counterfactual explanations and recourse (48 papers)
Generate actionable counterfactuals and recourse for decisions of trained models.
Most cited and most cited since 2024:
- Axiomatic Attribution for Deep Networks (ICML 2017 · 2,911 citations)
- Understanding Black-box Predictions via Influence Functions (ICML 2017 · 1,328 citations)
- Shaping Up SHAP: Enhancing Stability through Layer-Wise Neighbor Selection (AAAI 2024 · 18 citations)
- Beyond TreeSHAP: Efficient Computation of Any-Order Shapley Interactions for Tree Ensembles (AAAI 2024 · 16 citations)
Decision trees · 201 papers
Keywords: trees, decision tree, forests
Approaches in this cluster:
- Tree ensemble optimization (92 papers)
Improve, prune and interpret random forests and boosted tree ensembles. - Optimal decision tree learning (82 papers)
Compute optimal or sparse decision trees with algorithmic and generalization guarantees. - Compact interpretable model classes (27 papers)
Learn small interpretable models such as diagrams, lattices and oblique trees.
Most cited and most cited since 2024:
- LightGBM: A Highly Efficient Gradient Boosting Decision Tree (NeurIPS 2017 · 9,433 citations)
- Neural Additive Models: Interpretable Machine Learning with Neural Nets (NeurIPS 2021 · 121 citations)
- Symbolic Regression Enhanced Decision Trees for Classification Tasks (AAAI 2024 · 15 citations)
- Generative Model for Decision Trees (AAAI 2024 · 5 citations)
Related topics in Learning theory and algorithms
- Matrix and tensor methods (1,467)
- Sample complexity (1,553)
- Combinatorial and search optimization (1,576)
- Fairness (550)
- Submodular and game algorithms (1,064)
- Kernels and regression theory (1,272)
- Clustering (998)
- Optimal transport (462)
