Research map / Image generation / Autoregressive and latent generation
Linear combinations of latents in generative models: subspaces and beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LOL, a closed-form transport map that rescales linear combinations of Gaussian latents so they again follow the latent distribution, improving interpolation, centroids, and low-dimensional latent subspaces for diffusion and flow matching models.
Problem
Linear combinations of latent vectors (interpolation, centroids, subspaces) leave the latent distribution the generative model expects, causing implausible generations; existing fixes (SLERP, NAO) are costly, tuned, or limited to interpolation and Gaussian latents.
Approach
For Gaussian latents, a linear combination with weights w has mean alpha*mu and covariance beta*Sigma; an affine rescaling T(y) = (1 - alpha/sqrt(beta))mu + y/sqrt(beta) restores N(mu, Sigma) and is the Monge optimal map. It generalises to any linear combination (including subspace projections via Moore-Penrose weights) and to independent non-Gaussian latents via CDF transforms.
Key results
- On interpolation, LOL reaches 67.4% class accuracy and FID 38.9 versus SLERP 64.6% / 42.6 and NAO 62.1% / 46.0 on 1250 ImageNet pairs with SD 2.1.
- On centroid determination, LOL achieves FID 87.7 and 46.3% accuracy, outperforming Euclidean (FID 310.4) and NAO (FID 93.0).
- Empirical check on unit Gaussians: 99% confidence interval of beta is [0.9934, 1.0067] for SD3 (D=147456) and [0.9868, 1.014] for SD2.1 (D=36864).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SLERP with LOL for interpolation | Stable Diffusion 2.1 · ImageNet1k interpolation | accuracy: 64.56% -> 67.38666666666666% | Helped |
| Replace SLERP with LOL for interpolation | Stable Diffusion 2.1 · ImageNet1k interpolation | FID: 42.5827 -> 38.8714 | Helped |
| Replace NAO with LOL for interpolation | Stable Diffusion 2.1 · ImageNet1k interpolation | time: $30$s -> $6e^{-3}$s | Helped |
| Replace the best baseline, Standardised Euclidean, with LOL for centroid determination | Stable Diffusion 2.1 · ImageNet1k centroid determination | accuracy: 44.571428571428573% -> 46.285714285714286% | Helped |
| Replace the best baseline, Mode norm Euclidean, with LOL for centroid determination | Stable Diffusion 2.1 · ImageNet1k centroid determination | FID: 88.37776087449538 -> 87.71343603361333 | Helped |
| Replace NAO with LOL for centroid determination | Stable Diffusion 2.1 · ImageNet1k centroid determination | time: $90$s -> $6e^{-4}$s | Helped |
Surprising findings
- Shapiro-Wilk, often regarded as a powerful normality test, performed poorly in this setting, giving high p-values for several failure cases.
- NAO performed worse than spherical interpolation in this setup, contrary to the original NAO paper's reported results.
Limitations
- Transformed combinations only follow the target distribution if the seed latents do; latents from inversion may be invalid.
- Relies on existing normality tests that may be stricter than necessary for the network.
- Gaussian derivation for SLERP relies on large-D approximations; general-latent extension assumes independent elements with known CDF.
Experimental setup
Datasets: ImageNet1k ShapeNet
Benchmarks: ImageNet1k
Metrics: FID class prediction accuracy Kolmogorov-Smirnov p-value LPIPS reconstruction error Q-Align visual quality wavedrag (CFD simulation)
Baselines: LERP SLERP Norm-Aware Optimisation Euclidean centroid Standardised Euclidean centroid Mode norm Euclidean centroid
Builds on: SLERP Norm-Aware Optimisation Denoising Diffusion Implicit Models Flow Matching
Closest papers by meaning
- Complexity Matters: Rethinking the Latent Space for Generative Modeling (NeurIPS 2023 · similarity 0.88)
- Lost in Latent Space: Examining failures of disentangled models at combinatorial generalisation (NeurIPS 2022 · similarity 0.87)
- Latent Zoning Network: A Unified Principle for Generative Modeling, Representation Learning, and Classification (NeurIPS 2025 · similarity 0.87)
- Stabilize the Latent Space for Image Autoregressive Modeling: A Unified Perspective (NeurIPS 2024 · similarity 0.87)
- Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution (ICML 2024 · similarity 0.86)
- GLAD: Improving Latent Graph Generative Modeling with Simple Quantization (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.