Research map / Learning theory and algorithms / Matrix and tensor methods
Robustness of Nonlinear Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that nonlinear ICA / representation learning with near-local-isometric mixing is approximately identifiable up to linear maps, with MCC bounds in terms of a non-isometry measure Θ, and that perturbed linear ICA recovers the mixing and sources approximately.
Problem
Identifiability results for nonlinear representation learning and ICA are brittle: exact identifiability for mixing functions does not imply robustness under slight misspecification of the function class. The paper asks whether identifiability survives when the mixing is only approximately a local i
Approach
Measure the distance of a mixing function to local isometries via a pointwise distance of its Jacobian (and its inverse) to SO(d); using rigidity results from elasticity theory, show that near-isometric mixings are close to affine maps, so the latents are identified up to linear transformations with error bounded by Θ. For perturbed linear ICA, show that local extrema of a contrast function persis
Key results
- Theoretical bound: for near-local-isometric mixing, MCC(Ŝ,S) ≥ 1 − C_3 Θ_p²(f,Ω) for the proposed estimator (learn maximally isometric g then linear ICA).
- For perturbed linear ICA x = As + ηh(s), there is a matrix W with |W − W̄| = O(η) and MCC(Ŝ,S) ≥ 1 − C_2 η².
- Toy experiment: a VAE initialized at the true piecewise-linear mixing of Gaussian latents drifts to an orthogonal transformation, showing non-robustness of analytic-style identifiability.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize a ReLU VAE with the true piecewise-linear mixing and its inverse, then train it | VAE · 2D Gaussian latent-variable toy setting | preservation of the true representation | Hurt |
Surprising findings
- Identifiability for piecewise linear / polynomial mixings can be non-robust: a VAE initialized at the true mixing learns a simple orthogonal transformation instead.
- Local isometries from R^d to R^d are affine (Liouville), and near-isometries are close to affine by a rigidity result, which is highly non-trivial since the local rotation may vary with s.
Limitations
- Results for the undercomplete case assume the observational distribution and manifold are exactly known.
- Optimization for the maximally isometric unmixing g is non-convex and hard to optimize in practice; existence of a minimizer not proven.
- Perturbed linear ICA results give only existence of local extrema and local convergence, not global or efficient recovery.
Experimental setup
Metrics: mean correlation coefficient (MCC)
Builds on: Identifying the latent space geometry of VAEs / orthogonal coordinate transforma Liouville's theorem on local isometries Geometric rigidity (Friesecke, James, Müller) Linear ICA identifiability
Closest papers by meaning
- Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability (NeurIPS 2025 · similarity 0.89)
- Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples (ICML 2024 · similarity 0.88)
- Synergy Between Sufficient Changes and Sparse Mixing Procedure for Disentangled Representation Learning (ICLR 2025 · similarity 0.86)
- Beyond ICA: Identifiability by Symmetry Breaking (NeurIPS 2026 · similarity 0.85)
- Multi-View Causal Representation Learning with Partial Observability (ICLR 2024 · similarity 0.85)
- I-Con: A Unifying Framework for Representation Learning (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.