Research map / Learning theory and algorithms / Prediction and decision losses

Theoretical Investigations and Practical Enhancements on Tail Task Risk Minimization in Meta Learning

NeurIPS 2024 · Learning theory and algorithms › Prediction and decision losses · approach: Distributionally robust optimization · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Recasts the two-stage CVaR-based distributionally robust meta learning (DR-MAML) as a max-min / Stackelberg game, proves convergence and generalization bounds, and replaces crude Monte Carlo VaR estimation with kernel density estimation (DR-MAML+).

Problem

Tail task risk minimization in meta learning (DR-MAML) lacks a solution concept, algorithmic/convergence understanding, and generalization analysis in the tail, and its crude Monte Carlo quantile estimates are inaccurate.

Approach

Model the two-stage tail-risk strategy as a Stackelberg game between a task-selection leader (max over uncertainty set of tail task distributions) and a sub-gradient follower (min over meta parameters); derive local Stackelberg equilibrium, convergence rate, CVaR performance gap and tail generalization bound; and replace crude MC VaR estimation with KDE quantiles.

Key results

• On mini-ImageNet 5-way 1-shot with alpha=0.5, DR-MAML+ reaches 49.9% average and 48.1% CVaR accuracy on meta-testing tasks, versus 49.4% and 47.5% for DR-MAML.
• On 5-shot sinusoid regression (alpha=0.7), DR-MAML+ consistently outperforms baselines on average and CVaR indicators, with smaller standard errors.
• Empirically, KDE yields more accurate VaR estimates than crude Monte Carlo, with a sharper decreasing error trend as the number of tasks grows.

Measured effects reported in the paper

Surprising findings

• The KDE-based DR-MAML+ is comparable to DR-MAML on mini-ImageNet, which the authors attribute to the small training batch size weakening KDE's advantage.
• Tail-risk methods like DR-MaPLe incur extra training time and memory because they evaluate double task batches to screen sub-batches.

Limitations

• Tail-risk training requires double task batches, raising computation and memory costs.
• Theoretical results rely on Lipschitz and boundedness assumptions inherited from prior work and the global equilibrium is NP-hard, so only local equilibria are analysed.
• Gains over DR-MAML are modest and the KDE advantage vanishes with small batch sizes.

Experimental setup

Datasets: mini-ImageNet sinusoid regression tasks pendulum system identification point robot navigation

Benchmarks: sinusoid regression pendulum system identification mini-ImageNet 5-way 1-shot point robot navigation

Metrics: average risk (MSE / accuracy / return) worst-case risk CVaR_alpha

Baselines: MAML TR-MAML DRO-MAML DR-MAML MaPLe DR-MaPLe

Builds on: DR-MAML MAML MaPLe TR-MAML DRO-MAML

Closest papers by meaning

• On the Stability and Generalization of Meta-Learning (NeurIPS 2024 · similarity 0.88)
• Fast and Robust: Task Sampling with Posterior and Diversity Synergies for Adaptive Decision-Makers in Randomized Environments (ICML 2025 · similarity 0.88)
• Fast Rate Bounds for Multi-Task and Meta-Learning with Different Sample Sizes (NeurIPS 2025 · similarity 0.86)
• Bayesian Nonparametrics Meets Data-Driven Distributionally Robust Optimization (NeurIPS 2024 · similarity 0.86)
• Learning Robust Neural Processes with Risk-Averse Stochastic Optimization (ICML 2025 · similarity 0.86)
• More Flexible PAC-Bayesian Meta-Learning by Learning Learning Algorithms (ICML 2024 · similarity 0.85)