Research map / Label-efficient and robust learning / Self-supervised and contrastive
Neural Modes: Self-supervised Learning of Nonlinear Modal Subspaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns nonlinear physics-based modal subspaces for real-time simulation by self-supervised minimization of elastic energy (Neural Modes), avoiding curated simulation data.
Problem
Learned nonlinear subspaces for real-time simulation are typically trained supervised on pre-computed simulation data with geometric losses, yielding high-energy configurations, overfitting, entangled latent dimensions, and poor generalization.
Approach
Extends nonlinear compliant modes to multi-dimensional modal spaces and trains an MLP to output constrained equilibrium corrections by directly minimizing the mechanical energy (with constraint penalty) over sampled modal coordinates, with no simulation dataset required.
Key results
- On the square sheet, Neural Modes (grid) achieve test average energy of 22730 vs 24007 for PCA+AE and 24325 for L2 Supervised, with test energy error of 215 vs 2259 for PCA+AE.
- Neural Modes do not overfit: test energy and stress keep decreasing during training while PCA+AE and L2 Supervised test losses increase.
- On Armadillo, Sharp et al.'s method collapses to effectively 3 dimensions (eigenvalues 2.3, 1.6, 1.1, 0, 0), while Neural Modes give near-orthogonal latent directions; simulation runs at about 35 ms per step.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the subspace by minimizing mechanical energy instead of using geometric L2 supervision (ablation) | Neural Modes versus L2 Supervised · Square sheet test set | average elastic energy error: 2074 -> 386 | Helped |
| train the subspace by minimizing mechanical energy instead of using geometric L2 supervision (ablation) | Neural Modes versus L2 Supervised · Square sheet test set | average elastic energy: 24325 -> 22891 | Helped |
| use regular grid sampling instead of stochastic sampling (ablation) | Neural Modes · Square sheet test set | average elastic energy error: 386 -> 245 | Helped |
| use regular grid sampling instead of stochastic sampling (ablation) | Neural Modes · Square sheet test set | maximum elastic energy error: 1154 -> 3062 | Mixed |
| use elastic energy rather than L2 loss as the early-stopping criterion (ablation) | PCA+AE · Square sheet test set | elastic energy: 24007 -> 23484 | Helped |
| use elastic energy rather than L2 loss as the early-stopping criterion (ablation) | PCA+AE (grid) · Square sheet test set | elastic energy: 23217 -> 23051 | Helped |
| use elastic energy rather than L2 loss as the early-stopping criterion (ablation) | L2 Supervised · Square sheet test set | elastic energy: 24325 -> 23682 | Helped |
| replace self-supervised Neural Modes with supervised PCA+AE | Neural Modes · Deformable solids test set | average elastic energy: 55.8 -> 987.5 (×104) | Hurt |
| add the mode-orthogonality regularizer | Neural subspace learning · Armadillo 5D subspace | effective dimensionality: eigenvalues (2.3, 1.6, 1.1, 0.0, 0.0) | Mixed |
Surprising findings
- Geometry-based supervised autoencoders (PCA+AE) show test energy increasing during training, i.e., overfitting for all loss functions examined.
- Sharp et al.'s energy-based unsupervised subspace suffers mode collapse despite its isometry regularizer.
Limitations
- Restricted to differentiable energy functions and constraints.
- Relies on L-BFGS optimization; simulation acceleration algorithms not investigated.
- Only evaluated on relatively small examples (square sheet, giraffe, Armadillo, bunny).
Experimental setup
Datasets: square sheet simulation samples (baseline training) Armadillo, giraffe, bunny tetrahedral meshes
Benchmarks: square sheet elastic energy test set Armadillo deformable solid test set full-space simulation comparison
Metrics: average elastic energy energy error stress error nodal force error correlation matrix eigenvalues runtime per time step
Baselines: PCA+AE (Latent-space Dynamics) L2 Supervised Sharp et al. unsupervised subspace Modal Analysis
Builds on: Nonlinear Compliant Modes Latent-space Dynamics linear modal analysis Sharp et al. unsupervised subspace learning
Closest papers by meaning
- Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems (CVPR 2025 · similarity 0.85)
- Universal Physics Transformers: A Framework For Efficiently Scaling Neural Operators (NeurIPS 2024 · similarity 0.85)
- Learning Physics Informed Neural ODEs with Partial Measurements (AAAI 2025 · similarity 0.84)
- Learning Macroscopic Dynamics from Partial Microscopic Observations (NeurIPS 2024 · similarity 0.84)
- Predicting the Energy Landscape of Stochastic Dynamical System via Physics-informed Self-supervised Learning (ICLR 2025 · similarity 0.84)
- Multiple Physics Pretraining for Spatiotemporal Surrogate Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.