Research map / Label-efficient and robust learning / Domain adaptation and generalization
Continuous Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Continuous Domain Generalization (CDG) and NeuralLio, a Lie-group-based neural transport operator that maps model parameters across multi-dimensional continuous domain descriptors, with gating and local charts for imperfect descriptors.
Problem
Generalizing predictive models to unseen domains defined by arbitrary combinations of continuous latent factors, beyond discrete domains or a single temporal axis, with noisy or incomplete descriptors.
Approach
Models the family of domain-optimal parameters as a low-dimensional embedded submanifold and learns a structure-preserving transport operator using exponentiated neural Lie algebra fields (geometric continuity plus group-like algebraic consistency), augmented with a descriptor gate and k-NN local charts.
Key results
- On 2-Moons NeuralLio reaches 3.2% error vs 13.1% for the best baseline (ERM-D); on MNIST 9.5% vs 26.3% (NDA).
- On Traffic NeuralLio achieves MAE 15.1 vs 15.9 for ERM-D; on Yearbook 4.8% error vs 6.6% for Koodos.
- Learned operator satisfies identity, associativity and invertibility with cosine similarity of 99.9%, 99.1% and 98.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NeuralLio instead of ERM-D | Continuous domain generalization · 2-Moons | error rate (%): 13.1 ± 1.5 -> 3.2 ± 1.2 | Helped |
| Use NeuralLio instead of NDA | Continuous domain generalization · MNIST | error rate (%): 26.3 ± 0.7 -> 9.5 ± 1.1 | Helped |
| Use NeuralLio instead of DANN | Continuous domain generalization · fMoW | error rate (%): 26.0 ± 0.7 -> 24.5 ± 0.5 | Helped |
| Use NeuralLio instead of CIDA | Continuous domain generalization · ArXiv | error rate (%): 35.3 ± 0.4 -> 34.7 ± 0.4 | Helped |
| Use NeuralLio instead of Koodos | Continuous domain generalization · Yearbook | error rate (%): 6.6 ± 1.3 -> 4.8 ± 0.3 | Helped |
| Use NeuralLio instead of ERM-D | Continuous domain generalization · Traffic | MAE: 15.9 ± 0.1 -> 15.1 ± 0.1 | Helped |
| Add geometric and algebraic constraints to transport instead of using a plain MLP (ablation) | NeuralLio · 2-Moons | error rate (%): 13.3 ± 0.4 -> 3.2 ± 1.2 | Helped |
| Add geometric and algebraic constraints to transport instead of using a plain MLP (ablation) | NeuralLio · Traffic | MAE: 16.5 ± 1.0 -> 15.1 ± 0.1 | Helped |
| Add the exponential mapping from Lie algebra elements to the Lie group (ablation) | NeuralLio · 2-Moons | error rate (%): 34.8 ± 0.5 -> 3.2 ± 1.2 | Helped |
| Add the exponential mapping from Lie algebra elements to the Lie group (ablation) | NeuralLio · Traffic | MAE: 16.4 ± 0.3 -> 15.1 ± 0.1 | Helped |
| Add descriptor gating (ablation) | NeuralLio · Traffic | MAE: 16.3 ± 0.4 -> 15.1 ± 0.1 | Helped |
| Add local chart-based inference (ablation) | NeuralLio · Traffic | MAE: 16.1 ± 0.3 -> 15.1 ± 0.1 | Helped |
Surprising findings
- Temporal domain generalization methods (TKNets, DRAIN, Koodos) are not applicable beyond ordered domains and fail when the descriptor space lacks ordering.
- Naively adding descriptors as input (ERM-D) can introduce spurious correlations that hurt generalization on some datasets, e.g., ArXiv where it is worse than ERM.
Limitations
- No existing method, including NeuralLio, achieves reasonable accuracy on the synthetic 2-Moons dataset in some baselines' comparisons and gains on ArXiv and fMoW are small or absent.
- Theoretical result assumes a complete, non-degenerate descriptor space that exists only theoretically; practical descriptors are approximations.
- Evaluation uses fixed architectures per dataset and experiments are limited to low-dimensional descriptors (2-3 dims in synthetic data).
Experimental setup
Datasets: 2-Moons MNIST fMoW ArXiv Yearbook Traffic
Benchmarks: 2-Moons MNIST fMoW ArXiv Yearbook Traffic
Metrics: error rate (%) MAE
Baselines: ERM IRM V-REx GroupDRO Mixup DANN MLDG CDANN URM ERM-D NDA CIDA
Builds on: Temporal Domain Generalization NeuralODE CIDA Koodos Physics-Informed Neural Networks
Closest papers by meaning
- Continuous Temporal Domain Generalization (NeurIPS 2024 · similarity 0.89)
- Generalizing across Temporal Domains with Koopman Operators (AAAI 2024 · similarity 0.87)
- Enhancing Evolving Domain Generalization through Dynamic Latent Representations (AAAI 2024 · similarity 0.87)
- Weight Diffusion for Future: Learn to Generalize in Non-Stationary Environments (NeurIPS 2024 · similarity 0.86)
- Robust Domain Generalization under Divergent Marginal and Conditional Distributions (NeurIPS 2026 · similarity 0.86)
- Out-of-Domain Generalization in Dynamical Systems Reconstruction (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.