Research map / Probabilistic and scientific modeling / Variational and generative sampling
A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives a statistical error analysis of Wasserstein Autoencoders (W1 and MMD dissimilarities) showing excess risk rates depend on the intrinsic (Minkowski) dimension of the data rather than the ambient dimension.
Problem
Existing statistical guarantees for VAEs/WAEs ignore that natural data lies near low-dimensional structure and use fixed architectures, so their bounds are inefficient in high ambient dimension.
Approach
Derives an oracle inequality decomposing WAE excess risk into misspecification, generalization and optimization errors, and chooses ReLU encoder/generator sizes as functions of n so that rates scale with the upper Minkowski dimension of the data support, yielding roughly n^{-1/(2+d_mu)} for Lipschitz models.
Key results
- Theoretical excess risk rate of order n^{-1/(2+d_mu)} (up to log factors) for Lipschitz true models with W1 or MMD dissimilarity.
- Proof-of-concept experiment shows lower FID and reconstruction error for intrinsic dimension 2 than for 16 on synthetic images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reduce intrinsic dimension from d_int = 16 to d_int = 2 | WAE-GAN and WAE-MMD · Synthetic soap-bubble images | FID | Helped |
| reduce intrinsic dimension from d_int = 16 to d_int = 2 | WAE-GAN and WAE-MMD · Synthetic soap-bubble images | reconstruction error | Helped |
Limitations
- Assumes realizability with a single-chart smooth true encoder and its smooth left inverse, a strong assumption.
- Optimality (minimax) of the rates is not established.
- Optimization error is not analyzed; regularity assumptions on learned networks are not guaranteed by the analysis.
Experimental setup
Datasets: ImageNet (for pre-training BiGAN) synthetic soap-bubble images generated by BiGAN decoder
Benchmarks: synthetic soap-bubble images (proof-of-concept)
Metrics: Frechet Inception Distance reconstruction error
Baselines: f-WAE analysis GAN excess risk rates
Builds on: Wasserstein Autoencoders WGAN Weed and Bach (2019) manifold rates
Closest papers by meaning
- Statistical Regeneration Guarantees of the Wasserstein Autoencoder with Latent Space Consistency (NeurIPS 2021 · similarity 0.93)
- Wasserstein Auto-Encoders (ICLR 2018 · similarity 0.90)
- Gromov-Wasserstein Autoencoders (ICLR 2023 · similarity 0.90)
- StrWAEs to Invariant Representations (ICML 2024 · similarity 0.89)
- Shape your Space: A Gaussian Mixture Regularization Approach to Deterministic Autoencoders (NeurIPS 2021 · similarity 0.89)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.