Research map / Optimization / Implicit bias and generalization

Drifting Fields are not Conservative

NeurIPS 2026 · Optimization › Implicit bias and generalization · approach: Gradient flow and mean-field dynamics · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows drifting-model drift fields are generally non-conservative (nonzero curl) due to position-dependent normalization, and proposes sharp normalization with a sharp kernel k# that yields a conservative field and a scalar log-KDE loss, performing comparably on MNIST, Fashion-MNIST and ImageNet latent generation.

Problem

Whether drifting models, which train a generator by following a vector-valued drift field rather than minimizing a scalar loss, are equivalent to scalar-loss optimization; the drift field is generally not conservative.

Approach

Identifies the position-dependent normalization as the source of non-conservatism and introduces the sharp kernel k# (antiderivative of k) to normalize by sharp KDEs, giving a conservative sharp-normalized drift field equal to a score difference of KDEs and a scalar log-KDE loss.

Key results

• On MNIST, the Gaussian log-KDE objective reaches FID 7.21 versus 37.62 for the Gaussian drifting objective.
• On ImageNet ablation setting, log-KDE with Gaussian kernel reaches FID 9.43 and Laplace sharp reaches 9.35, versus reported drifting FID 8.67 with a single scale.
• On Fashion-MNIST, Gaussian log-KDE reaches FID 30.57 versus 64.87 for Drifting-Gauss.

Surprising findings

• The original drifting implementation computes a different normalization than the stated drift field, and a corrected naive variant outperforms it on MNIST.
• The Gaussian kernel is the unique radial kernel for which the drift field and sharp-normalized field coincide.

Limitations

• Population-level theory does not directly imply convergence for minibatch KDE estimates and neural generators.
• Performance remains bandwidth-sensitive; log-KDE needs larger bandwidths on ImageNet.
• Experiments are limited to small DiT models on image benchmarks with one seed on ImageNet.

Experimental setup

Datasets: MNIST Fashion-MNIST ImageNet

Benchmarks: MNIST Fashion-MNIST ImageNet

Metrics: FID-50k F1 (precision/recall)

Baselines: Drifting-Gauss Drifting-Laplace Naive-Laplace MMD gradient

Builds on: drifting models maximum mean discrepancy Wasserstein gradient flow denoising score matching

Closest papers by meaning

• Kernel-Gradient Drifting Models (NeurIPS 2026 · similarity 0.88)
• On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models (ICLR 2024 · similarity 0.88)
• Analytical Correction for Subsampling Bias in Drifting Models (NeurIPS 2026 · similarity 0.87)
• Generative Modeling via Drifting (NeurIPS 2026 · similarity 0.85)
• Guidance with Spherical Gaussian Constraint for Conditional Diffusion (ICML 2024 · similarity 0.85)
• Importance Corrected Neural JKO Sampling (ICML 2025 · similarity 0.84)