Research map / Label-efficient and robust learning / Domain adaptation and generalization
Continuous Invariance Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Continuous Invariance Learning (CIL), a min-max adversarial method that learns invariant features across continuously indexed domains by aligning E^y[t|Φ(x)] across classes instead of E^t[y|Φ(x)] across domains.
Problem
Invariance learning methods assume discrete domain indices, but many real tasks (e.g., time-indexed cloud CPU prediction) have continuous domains; existing methods fail when there are many domains with few samples each.
Approach
Replace domain-wise alignment of E^t[y|Φ(x)] with class-wise alignment of E^y[t|Φ(x)]: two domain-index regressors (one using Φ(x) only, one using Φ(x) and y) are trained adversarially, and the featurizer is penalized when their losses differ, enforcing conditional independence of y and t given Φ(x).
Key results
- On continuous CMNIST (1000 domains), CIL reaches 57.20% OOD accuracy with linear shift and 59.50% with sine shift, vs. REx ~41.9% and 42.2%.
- On Alipay auto-scaling, CIL achieves 71.29% OOD accuracy (L2) vs. 57.38% for ERM and 66.49% for the best discrete baseline (IRMv1 at 66.49%).
- On WildTime Yearbook, CIL reaches 71.22% OOD accuracy, about 1.5% above prior SOTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CIL instead of the strongest reported baseline | Continuous CMNIST Linear · Continuous CMNIST Linear OOD | accuracy: 41.87 (0.48) -> 57.20 (6.89) | Helped |
| use CIL instead of the strongest reported baseline | Continuous CMNIST Sine · Continuous CMNIST Sine OOD | accuracy: 42.24 (0.74) -> 59.50 (8.67) | Helped |
| use CIL instead of the strongest reported baseline | Continuous CMNIST with 1024 domains · Continuous CMNIST, 1024 domains | accuracy: 58.3 -> 67.2 | Helped |
| use CIL instead of the strongest reported baseline | HousePrice · HousePrice OOD | accuracy: 77.54 (0.30) -> 79.29 (0.77) | Helped |
| use CIL instead of the strongest reported baseline | Insurance Fraud · Insurance Fraud OOD | accuracy: 73.52 (0.46) -> 76.54 (1.03) | Helped |
| use CIL instead of the strongest reported baseline | Alipay Auto-scaling · Alipay Auto-scaling OOD | accuracy: 66.20 (0.06) -> 71.29 (0.04) | Helped |
| use CIL | Yearbook · Yearbook worst test OOD domain | accuracy: 71.22% | Helped |
| use L2 domain regression loss instead of L1 (ablation) | CIL on HousePrice · HousePrice OOD | accuracy: 77.98 (1.02) -> 79.29 (0.77) | Helped |
| use L1 domain regression loss instead of L2 (ablation) | CIL on Insurance Fraud · Insurance Fraud OOD | accuracy: 75.01 (1.18) -> 76.54 (1.03) | Helped |
| use L1 domain regression loss instead of L2 (ablation) | CIL on Alipay Auto-scaling · Alipay Auto-scaling OOD | accuracy: 71.29 (0.04) -> 68.51 (1.33) | Hurt |
Limitations
- Min-max optimization can reach suboptimal solutions.
- Experiments use discretized time points as continuous domains rather than truly continuous domain indices.
- Assumes the label variable is discrete (classification).
Experimental setup
Datasets: CMNIST HousePrice (Kaggle) Insurance Fraud (Kaggle) Alipay Auto-scaling (proprietary) WildTime Yearbook
Benchmarks: Continuous CMNIST HousePrice Insurance Fraud Alipay Auto-scaling WildTime Yearbook
Metrics: accuracy
Baselines: ERM IRMv1 REx GroupDRO Diversify IIBNet InvRAT EIIL HRM ZIN
Builds on: Invariant Risk Minimization REx IRMv1 CIDA
Closest papers by meaning
- Learning Conditional Invariances through Non-Commutativity (ICLR 2024 · similarity 0.85)
- Rethinking Invariance in In-context Learning (ICLR 2025 · similarity 0.85)
- Empowering Graph Invariance Learning with Deep Spurious Infomax (ICML 2024 · similarity 0.84)
- Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach (AAAI 2024 · similarity 0.84)
- Continuous Domain Generalization (NeurIPS 2025 · similarity 0.84)
- How Data Mixing Shapes In-Context Learning: Asymptotic Equivalence for Transformers with MLPs (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.