Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Learning Infinitesimal Generators of Continuous Symmetries from Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns continuous symmetries as infinitesimal generators (MLP vector fields integrated by Neural ODE) by minimizing a differentiable validity score, applied to CIFAR-10 image affine symmetries and PDE symmetries used for FNO data augmentation.
Problem
Discovering continuous symmetries of data (images, PDEs) without prior knowledge of Lie group structure, beyond linear or affine transformations.
Approach
Model one-parameter groups as flows of learnable MLP vector fields (Neural ODE) and optimize a task-specific validity score (feature cosine similarity for classifiers, PDE residual for PDEs) under orthonormality and Lipschitz regularizers, so nonlinear and approximate symmetries can be found end-to-end.
Key results
- On CIFAR-10 the learned augmentation reaches 94.9 ± 0.1% test accuracy versus 95.1 ± 0.1% for default augmentation and 92.4 ± 0.3% for no augmentation (ResNet-18).
- The method recovers six affine generators in the first six slots from CIFAR-10 (two translations found first), and the three ground-truth symmetries for KdV, KS, and Burgers PDEs.
- For FNO on KdV with 2^9 data, learned-symmetry augmentation reduces test error from 0.398 to 0.0824 (ground-truth symmetries: 0.0640).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learned symmetry augmentation instead of no augmentation | ResNet-18 · CIFAR-10 test | accuracy (%): 92.4\pm 0.3 -> 94.9 \pm 0.1 | Helped |
| Use learned symmetry augmentation instead of no augmentation | FNO · KdV, 2^9/2^7/2^5 data | Test NMSE: 0.398\pm0.013 -> 0.0824 \pm 0.0052; 1.42 \pm 0.05 -> 0.283 \pm 0.018; 4.47 \pm 0.26 -> 1.12 \pm 0.10 | Helped |
| Use learned symmetry augmentation instead of no augmentation | FNO · KS, 2^9/2^7/2^5 data | Test NMSE: 0.00693 \pm 0.00039 -> 0.00614 \pm 0.00051; 0.324 \pm 0.031 -> 0.0422 \pm 0.0002; 5.78 \pm 0.13 -> 1.37 \pm 0.04 | Helped |
| Use learned symmetry augmentation instead of no augmentation | FNO · nKdV, 2^9/2^7/2^5 data | Test NMSE: (3.52\pm0.11)\times 10^{-3} -> (2.58 \pm 0.17)\times 10^{-3}; (3.16\pm0.06)\times 10^{-2} -> (2.96 \pm 0.50)\times 10^{- | Helped |
| Use learned symmetry augmentation instead of no augmentation | FNO · cKdV, 2^5 data | Test NMSE: (7.90\pm1.21)\times 10^{-4} -> (2.99 \pm 0.37)\times 10^{-4} | Helped |
| Add the approximate symmetry to LPS augmentation | FNO · cKdV, 2^5 data | Test NMSE: (5.90 \pm 0.17)\times 10^{-4} -> (3.70 \pm 0.15)\times 10^{-4} | Helped |
| Use bilinear interpolation instead of Whittaker-Shannon interpolation (ablation) | FNO augmentation · KS, 512 training data | Test NMSE: 0.006143 \pm 0.00051 -> 0.542 \pm 0.051 | Hurt |
| Replace WENO with bilinear interpolation followed by discrete differentiation (ablation) | Symmetry learning · KS | symmetry recovery | Hurt |
| Vary the Lipschitz threshold (ablation) | Symmetry learning · KS | ground-truth symmetry recovery | No clear effect |
| Use learned symmetry augmentation instead of ground-truth symmetry augmentation | FNO · KdV and KS, all reported data sizes | Test NMSE: KdV: 0.0640 \pm 0.0050 vs 0.0824 \pm 0.0052; 0.246 \pm 0.012 vs 0.283 \pm 0.018; 0.980 \pm 0.039 vs 1.12 \pm 0.10. KS: 0 | Hurt |
Surprising findings
- Translations are found as the most invariant one-parameter groups on CIFAR-10 and the ResNet trained without augmentation already appears insensitive to affine transformations.
- Bilinear interpolation with discrete differentiation fails to learn orthogonal symmetries; WENO differentiation is needed for gradient flow.
Limitations
- Requires careful choice of numerical differentiation and interpolation for stable training.
- ODE integration for augmentation generation is computationally expensive compared to standard augmentation.
- Only image affine symmetries and Lie point symmetries of 1D evolution PDEs on periodic domains are studied.
Experimental setup
Datasets: CIFAR-10 KdV equation simulations KS equation simulations Burgers equation simulations nKdV equation simulations cKdV equation simulations
Benchmarks: CIFAR-10 KdV KS Burgers nKdV cKdV
Metrics: test accuracy test NMSE inner product with ground-truth generators affine-ness
Baselines: Augerino LieGAN LieGG L-conv Lie point symmetry augmentation default augmentation (flip and crop)
Builds on: Neural ODE LieGAN LieGG L-conv Augerino
Closest papers by meaning
- A Generative Model of Symmetry Transformations (NeurIPS 2024 · similarity 0.92)
- Symmetry Discovery Beyond Affine Transformations (NeurIPS 2024 · similarity 0.90)
- Explicit Discovery of Nonlinear Symmetries from Dynamic Data (ICML 2025 · similarity 0.89)
- Lie Group Decompositions for Equivariant Neural Networks (ICLR 2024 · similarity 0.87)
- Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups (ICLR 2025 · similarity 0.87)
- AtlasD: Automatic Local Symmetry Discovery (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.