Research map / Label-efficient and robust learning / OOD and anomaly detection
Out-of-distribution Generalization for Total Variation based Invariant Risk Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends IRM-TV-l1 into a Lagrangian primal-dual model (OOD-TV-IRM/OOD-TV-Minimax) where the TV penalty is a learned function of the feature extractor and a dual parameter, optimized adversarially with a convergent subgradient scheme.
Problem
Out-of-distribution generalization within the IRM-TV framework remains unsolved because the TV penalty hyperparameter is inflexible and no tractable implementation of its feature-dependent variation exists.
Approach
Treat the TV penalty hyperparameter as a Lagrangian multiplier parameterized by a neural network of the feature extractor parameters (lambda(Psi, Phi)), yielding a minimax primal-dual game (semi-Nash equilibrium) solved by a convergent subgradient primal-dual algorithm.
Key results
- On simulation data, OOD-TV-IRM-l1 reaches mean 0.8944 and worst 0.8877 accuracy vs IRM-TV-l1 0.8713 / 0.8618.
- On House Prices, OOD-TV-Minimax-l1 achieves MSE 0.2621 mean vs ZIN 0.3203.
- On NICO Animal, OOD-TV-Minimax-l1 reaches 0.9112 vs ZIN 0.8541 accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · Simulation | mean accuracy: 0.8713 -> 0.8944 | Helped |
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · Simulation | worst accuracy: 0.8618 -> 0.8877 | Helped |
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · CelebA | mean accuracy: 0.7751 -> 0.7945 | Helped |
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · CelebA | worst accuracy: 0.7461 -> 0.7560 | Helped |
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · Landcover | mean accuracy: 0.6422 -> 0.6672 | Helped |
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · Adult | mean accuracy: 0.8298 -> 0.8435 | Helped |
| Replace the fixed IRM-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | IRM-TV-ℓ1 · House Price | mean MSE: 0.4009 -> 0.3383 | Helped |
| Replace the fixed Minimax-TV-ℓ1 penalty with an autonomous primal-dual TV penalty | Minimax-TV-ℓ1 · Colored MNIST | mean accuracy: 0.9322 -> 0.9514 | Helped |
| Replace ZIN with OOD-TV-Minimax-ℓ2 | ZIN · NICO Vehicle | mean accuracy: 0.7176 -> 0.7872 | Helped |
| Replace Minimax-TV-ℓ1 with OOD-TV-Minimax-ℓ1 | Minimax-TV-ℓ1 · NICO Animal | mean accuracy: 0.8289 -> 0.9112 | Helped |
Limitations
- Uses subgradient descent for non-differentiable gradient norm, with convergence theory relying on assumptions (bounded closed parameter set) that are only loosely justified.
- Evaluated on small-scale tasks with small MLP/CNN models; no large-scale pretrained model experiments.
- Semi-Nash equilibrium is a restricted notion, not full Nash equilibrium.
Experimental setup
Datasets: Simulation temporal data CelebA Landcover Adult House Prices Colored MNIST NICO
Benchmarks: Simulation temporal data CelebA Landcover Adult House Prices Colored MNIST NICO
Metrics: mean accuracy worst accuracy mean squared error STD over 10 repeats
Baselines: IRM ZIN IRM-TV-l1 Minimax-TV-l1 EIIL LfF TIVA
Builds on: Invariant Risk Minimization IRM-TV ZIN EIIL TIVA
Closest papers by meaning
- Pareto Invariant Risk Minimization: Towards Mitigating the Optimization Dilemma in Out-of-Distribution Generalization (ICLR 2023 · similarity 0.89)
- Invariant Risk Minimization Is A Total Variation Model (ICML 2024 · similarity 0.88)
- On the Connection between Invariant Learning and Adversarial Training for Out-of-Distribution Generalization (AAAI 2023 · similarity 0.87)
- Regularization Penalty Optimization for Addressing Data Quality Variance in OoD Algorithms (AAAI 2022 · similarity 0.87)
- Heterogeneous Risk Minimization (ICML 2021 · similarity 0.87)
- Towards Robust Out-of-Distribution Generalization Bounds via Sharpness (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.