Research map / Probabilistic and scientific modeling / Variational and generative sampling

Disentanglement as Identifiable Pushforward Factorisation

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Disentangled and discrete latent models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Defines disentanglement for smooth pushforward generative models as factorisation of the manifold density into seam factors, proves it is equivalent to two Jacobian-SVD conditions (C1-C2), links diagonal-posterior Gaussian VAEs to these conditions, and proves identifiability up to permutation and sign.

Problem

Disentanglement in VAEs and GANs lacks a formal, distribution-level definition, and it is unclear why diagonal posteriors and beta-VAEs promote it and whether generative factors are identifiable.

Approach

Define disentanglement as factorisation of the pushforward manifold density into 1-D seam factors aligned with latent axes; show this is equivalent to C1 (right singular vectors of the decoder Jacobian are standard basis vectors) and C2 (singular values depend only on their own latent coordinate); use an exact posterior-covariance identity to show diagonal posteriors encourage C1-C2 in expectation

Key results

• Diagonal-posterior Gaussian VAEs on dSprites show disentanglement (MIG and AAS) and derivative diagonality both broadly increasing with beta, while full-posterior VAEs show no clear trend.
• In the linear Gaussian case, only diagonal-covariance LVAEs drive right singular vectors of the decoder Jacobian to the identity, yielding axis-aligned disentangled components.
• The paper proves the manifold density factorises as prod_i p_i(z_i)/s_i(z) and that disentanglement holds iff C1-C2 hold almost everywhere.

Surprising findings

• Diagonal posteriors break the rotational symmetry of a Gaussian prior in linear VAEs, so the optimal decoder is identified up to permutation and sign.
• Disentanglement in the paper's framework is claimed to be encouraged only in aggregate or to first order, which may explain why VAE disentanglement is inconsistent in practice.

Limitations

• Results on identifiability and C1-C2 equivalence hold for smooth, injective (c.i.d.a.e.) generators on the regular set, and the VAE link is in expectation or to first order.
• Empirical support is limited to linear Gaussian data, dSprites and CelebA, with the full VAE argument resting on approximations.

Experimental setup

Datasets: dSprites CelebA synthetic Gaussian data

Benchmarks: dSprites CelebA

Metrics: mutual information gap (MIG) axis alignment score (AAS) derivative diagonality

Baselines: beta-VAE full-covariance VAE

Builds on: beta-VAE probabilistic PCA independent component analysis independent mechanism analysis

Closest papers by meaning

• When Is Unsupervised Disentanglement Possible? (NeurIPS 2021 · similarity 0.89)
• Disentangling by Factorising (ICML 2018 · similarity 0.88)
• Hyperprior Induced Unsupervised Disentanglement of Latent Representations (AAAI 2019 · similarity 0.87)
• Mechanistic Independence: A Principle for Identifiable Disentangled Representations (ICLR 2026 · similarity 0.87)
• Compositional Generalization via Forced Rendering of Disentangled Latents (ICML 2025 · similarity 0.86)
• Can Diffusion Models Disentangle? A Theoretical Perspective (NeurIPS 2025 · similarity 0.84)