Research map / Probabilistic and scientific modeling / Variational and generative sampling
On the Universality of Volume-Preserving and Coupling-Based Normalizing Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows volume-preserving normalizing flows are not universal under KL divergence and proposes a new distributional universality proof for well-conditioned affine coupling flows built layer by layer.
Problem
Theoretical expressivity of normalizing flows: whether volume-preserving flows and coupling-based flows (RealNVP-style) are distributional universal approximators, given prior proofs rely on ill-conditioned networks and compact supports.
Approach
Proves volume-preserving flows are not universal in KL and characterizes the distribution they learn instead; proves universality of well-conditioned affine coupling flows by iteratively appending coupling blocks that maximally reduce a latent-space KL-based loss improvement metric, which vanishes only at the standard normal.
Key results
- Proves volume-preserving normalizing flows are not universal under KL divergence via a 2D counterexample lower-bounding KL by total variation (Pinsker).
- Shows the KL-optimal volume-preserving flow is bounded below by KL(p*(z) || N(0, |Sigma|^{1/D} I)), with optimal C = |Sigma|^{-1/(2D)}.
- Proves distributional universality of L-bi-Lipschitz affine coupling flows with n blocks, with loss improvement Delta_affine converging to 0 without ill-conditioned networks and with global support.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use volume-preserving flows with constant Jacobian determinant | Normalizing flows · Arbitrary target distributions | KL divergence universality | Hurt |
| Add a single one-dimensional non-volume-preserving layer | Volume-preserving normalizing flows · Arbitrary target distributions | Distributional universality | Helped |
| Use well-conditioned affine coupling blocks with variable volume change | Affine coupling flows · Continuous targets with finite first and second moments and infinite support | Convergence to the standard normal | Helped |
| Use coupling functions more expressive than affine | Coupling flows · Layer-wise training | Loss improvement per block | Helped |
| Use more expressive coupling functions to fit non-Gaussian conditionals | Coupling flows · Non-Gaussian conditional distributions | Loss decrease | Helped |
Surprising findings
- The affine-coupling universality construction by Teshima et al. effectively builds a volume-preserving flow, which is not universal in KL divergence.
- Volume-preserving flows with a continuous map inherit the latent mode structure, so they cannot represent multimodal distributions with separated modes.
Limitations
- Universality proof is constructive layer-wise and gives no guarantee on the number of coupling blocks needed.
- Convergence metric Delta_affine is not shown to be directly equivalent to KL convergence in practice; joint end-to-end optimization not exploited.
- Practical fixes for volume-preserving flows (learned radial distribution, multimodal latent) are left to future work.
Experimental setup
Metrics: KL divergence loss improvement by affine coupling block
Baselines: Teshima et al. universality proof Koehler et al. universality proof NICE GIN
Builds on: RealNVP NICE GIN Neural Spline Flows
Closest papers by meaning
- Representational aspects of depth and conditioning in normalizing flows (ICML 2021 · similarity 0.90)
- Whitening Convergence Rate of Coupling-based Normalizing Flows (NeurIPS 2022 · similarity 0.89)
- Densely connected normalizing flows (NeurIPS 2021 · similarity 0.88)
- Coupling-based Invertible Neural Networks Are Universal Diffeomorphism Approximators (NeurIPS 2020 · similarity 0.88)
- Normalizing Flows are Capable Models for Continuous Control (NeurIPS 2025 · similarity 0.85)
- An Error Analysis of Flow Matching for Deep Generative Modeling (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.