Research map / Probabilistic and scientific modeling / Variational and generative sampling

Expressive Score-Based Priors for Distribution Matching with Geometry-Preserving Regularization

ICML 2025 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Score-based generative modeling analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a variational distribution-matching model whose prior is a score-based model learned via denoising score matching, using a Score Function Substitution trick so the encoder gradient needs only the prior's score.

Problem

Distribution matching for domain-invariant representation learning: fixed-form priors in likelihood-based (VAE) DM bias the latent space, while explicit flow priors are hard to train and adversarial DM is unstable.

Approach

Gradient-based VAUB training only needs the prior's score function, not its density, so the prior is parameterized by a denoising-score-matching model and the cross-entropy gradient is computed via a Score Function Substitution trick with detached latents, avoiding backprop through the score network; a CLIP-based Gromov-Wasserstein regularizer is added to preserve semantic geometry.

Key results

• On MNIST to USPS, the GW-SP variant reaches 96.1% accuracy versus 77.1% for DANN and 89.4% for ADDA; USPS to MNIST reaches 97.4%.
• On CelebA black-to-blonde hair translation, GW-SP improves top-1 retrieval to 9.0% from 5.0% without GW and raises SSIM to 0.542 from 0.393.
• Compared with LSGM, the SFS-based method uses up to 1.65x less VRAM and trains 1.02x to 1.13x faster across latent dimensions 8 to 256; LSGM diverges at sigma_min=0.001 while SFS remains stable.

Measured effects reported in the paper

Surprising findings

• LSGM becomes unstable at sigma_min=0.001 with diverging NLL, and its NLL at sigma_min=0.1 and 0.2 is better than at 0.01.
• Euclidean-space GW regularization (GW-EP) performs worse than no GW loss for image retrieval on CelebA.

Limitations

• Domain translation results use relatively simple VAE and diffusion networks, so image quality is not state of the art.
• Alternating bi-level optimization is a simple heuristic; more principled bi-level methods are left to future work.
• Computational speedup over LSGM is modest (about 1.02x to 1.13x) at the latent dimensions tested.

Experimental setup

Datasets: MNIST USPS UCI Adult CelebA FairFace Shape3D dSprites

Benchmarks: MNIST to USPS USPS to MNIST UCI Adult demographic parity CelebA image retrieval CelebA SSIM/LPIPS

Metrics: accuracy demographic parity gap domain adaptation accuracy top-k retrieval accuracy SSIM LPIPS AUROC NLL VRAM time per epoch

Baselines: ADDA DANN VAUB LSGM FCRL CVIB LAFTR-DP Unfair Classifier

Builds on: VAUB LSGM Gromov-Wasserstein distance denoising score matching

Closest papers by meaning

• Iterative Feature Matching: Toward Provable Domain Generalization with Logarithmic Environments (NeurIPS 2022 · similarity 0.87)
• Diffusion Model for Dense Matching (ICLR 2024 · similarity 0.86)
• Training Diffusion-based Generative Models with Limited Data (ICML 2025 · similarity 0.85)
• Local Curvature Smoothing with Stein's Identity for Efficient Score Matching (NeurIPS 2024 · similarity 0.85)
• Learning normalized image densities via dual score matching (NeurIPS 2025 · similarity 0.85)
• Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation (NeurIPS 2025 · similarity 0.85)