Research map / Probabilistic and scientific modeling / Variational and generative sampling

Hessian Geometry of Latent Space in Generative Models

ICML 2025 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Score-based generative modeling analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Reconstructs the Fisher metric of latent spaces by learning the log-partition function from posterior distributions over latents, validated on Ising/TASEP and applied to 2D slices of Stable Diffusion to reveal fractal phase boundaries.

Problem

Understanding the latent space geometry of generative models (statistical physics models and diffusion models), including abrupt changes in generated images under interpolation and the Fisher information metric for stochastic generators where prior pullback methods do not apply.

Approach

Approximate the posterior over latent parameters given generated samples (via a U2-Net or CLIP-distance-based approximation), then train an MLP log-partition function whose induced normalized distribution matches this posterior under JS divergence; the Hessian of the learned log Z gives the Fisher metric, with theory showing posterior^(1/N) converges to exp(-Bregman divergence).

Key results

• On Ising and TASEP, the convex log-partition method achieves lower free energy RMSE than Mean-as-Stat and PCA-VAE (e.g., Ising F RMSE 0.0883 vs 0.0981 and 0.1669; TASEP F RMSE 0.0112 vs 0.0529 and 0.0524).
• On Stable Diffusion slices, geodesics are approximately linear within a phase but deviate at phase boundaries, where the learned log Z has abrupt derivative changes.
• In deterministic sampling, the geodesic method gives CLIP length 72.3 vs 73.6 for linear interpolation and PPL 3.12 vs 3.17, with mean curvature 0.367 vs 0.00 for linear.

Measured effects reported in the paper

Surprising findings

• Diffusion latent space exhibits fractal phase boundaries, with samples from neighboring phases blending together and sensitivity persisting down to float16 precision.
• Using U2-Net to predict latent parameters yields a smooth log Z for the UNet case, while CLIP-based posterior yields sharp boundaries, indicating the learned geometry depends on the feature extractor.

Limitations

• Method is restricted to two-parameter (2D) slices of the latent space for practical reconstruction.
• Posterior approximation depends on feature extractor choice (CLIP distance vs U2-Net), and theoretical guarantees are for exponential families while diffusion analysis is empirical.
• Mean curvature for the proposed geodesics has high variance (0.367 ± 0.691), and deterministic-sampling comparisons show only parity with baselines.

Experimental setup

Datasets: Ising model samples TASEP samples Stable Diffusion generated images

Benchmarks: Ising model exact free energy TASEP exact free energy CLIP length pixel length Perceptual Path Length

Metrics: RMSE path length mean curvature

Baselines: Mean-as-Stat PCA-VAE Geodesic (Wang/Shao) Linear interpolation

Builds on: Bryant-Amari-Armstrong theorem U2-Net DDIM Perceptual Path Length

Closest papers by meaning

• Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion (ICLR 2025 · similarity 0.88)
• Isometric Representation Learning for Disentangled Latent Space of Diffusion Models (ICML 2024 · similarity 0.87)
• Probing the Latent Hierarchical Structure of Data via Diffusion Models (ICLR 2025 · similarity 0.85)
• Wasserstein Convergence of Critically Damped Langevin Diffusions (NeurIPS 2025 · similarity 0.85)
• Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models (NeurIPS 2024 · similarity 0.85)
• Dimension-free Score Matching and Time Bootstrapping for Diffusion Models (NeurIPS 2025 · similarity 0.85)