Research map / Image generation / Fast diffusion sampling

Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity

NeurIPS 2025 · Image generation › Fast diffusion sampling · approach: Diffusion theory and design space · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that Euclidean diffusion score functions on manifold data have multiscale singularity (normal component O(1/sigma^2), tangential O(1)) and proposes Niso-DM (non-isotropic normal noise) and Tango-DM (tangential-only loss with annealed Langevin sampling) to improve direct sampling.

Problem

Directly applying Euclidean VESDE diffusion models to data on a known submanifold suffers from score-function singularity with mismatched scales between tangential and normal components, causing manifold overfitting and inaccurate samples.

Approach

Decompose the perturbed score into tangential and normal parts; add extra noise along the normal direction (Niso-DM) to shrink the normal score scale from O(1/sigma^2) to O(1/sigma^{2alpha}), and train only the tangential score at small noise levels (Tango-DM), sampling with manifold-constrained annealed Langevin dynamics.

Key results

• On Spot the Cow and Stanford Bunny, Niso+res with Reverse SDE achieves the best JS divergence (3.48e-1 bunny, 3.28e-1 cow) versus Iso+res at 3.52e-1 and 3.30e-1.
• On SO(10), Niso+res with Reverse SDE reduces sliced 1-Wasserstein distance to 4.60e-3, versus 9.49e-3 for Iso+res and 7.14e-3 for RSSM.
• On alanine dipeptide, Niso (Reverse) gives 2-Wasserstein 8.31e-2 versus 8.60e-2 for Iso, with Niso Annealing at 8.24e-2.

Measured effects reported in the paper

Surprising findings

• Rescaling (+res) hurts on alanine dipeptide, which the authors attribute to simultaneously rescaling tangential and normal components.
• Tango-DM is not compatible with the Reverse SDE sampler and is slower due to annealed Langevin sampling.

Limitations

• Requires explicit knowledge of the manifold definition (known constraint function xi), not applicable to implicitly represented manifolds such as images or language.
• Tango-DM is incompatible with Reverse SDE sampling and is slower due to annealed Langevin dynamics.
• Rescaling can hurt performance in some settings (alanine dipeptide).

Experimental setup

Datasets: synthetic hyperplane Gaussian mixture Stanford Bunny mesh eigenpair densities Spot the Cow mesh eigenpair densities synthetic multimodal distribution on SO(10) alanine dipeptide molecular configurations

Benchmarks: Stanford Bunny mesh JS divergence Spot the Cow mesh JS divergence SO(10) sliced 1-Wasserstein alanine dipeptide 2-Wasserstein hyperplane MMD

Metrics: MMD Jensen-Shannon divergence sliced 1-Wasserstein distance 2-Wasserstein distance

Baselines: Iso-DM (vanilla isotropic VESDE) Riemannian sliced score matching (RSSM)

Builds on: VESDE score-based generative modeling denoising score matching Riemannian score-based generative models Riemannian sliced score matching

Closest papers by meaning

• Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes (ICML 2024 · similarity 0.91)
• Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion (ICLR 2025 · similarity 0.91)
• Manifold Diffusion Fields (ICLR 2024 · similarity 0.90)
• Diffusion Models Encode the Intrinsic Dimension of Data Manifolds (ICML 2024 · similarity 0.89)
• Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.89)
• Efficient Diffusion Models for Symmetric Manifolds (ICML 2025 · similarity 0.89)