Research map / Probabilistic and scientific modeling / Variational and generative sampling

Analytical Correction for Subsampling Bias in Drifting Models

NeurIPS 2026 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Score-based generative modeling analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows the minibatch softmax-weighted centroid in drifting models has an O(1/n) self-normalization bias and proposes Analytical Bias Correction (ABC), a closed-form plug-in fix that reduces bias to O(1/n^2) and improves CIFAR-10 FID at small n.

Problem

Drifting models approximate softmax-weighted attractive and repulsive centroids over full data and generator distributions with minibatches of size n; the self-normalized minibatch centroid is biased, and this bias degrades generation quality at small n.

Approach

Derive the leading O(1/n) bias of the self-normalized softmax centroid and correct it with a closed-form plug-in term sum_i alpha_i^2 (T_n - y_i) computed from the same minibatch, giving a corrected centroid that is still a convex combination of minibatch points.

Key results

• On CIFAR-10, ABC improves last-5-average FID at every tested n, e.g. 7.31 to 6.40 at n=8 and 4.34 to 3.89 at n=32.
• At n=8 with FID <= 10, ABC needs about 3.38M positive samples versus 8.19M for the standard estimator.
• Toy experiments show bias slopes of about -1 for the standard centroid and -2 for ABC on log-log axes.

Surprising findings

• Lowering the kernel temperature cannot substitute for the correction, per the compensation analysis.
• Epoch wall time with torch.compile is within measurement noise of the standard method (37.0s vs 38.0s fused).

Limitations

• Gains on CIFAR-10 are small at n=16 and n=64 and within seed variance; best FID at n=64 is worse for ABC (3.40 vs 3.24).
• Bias theorems assume bounded kernel weights and features, and the higher-order toy results show ABC and standard nearly overlapping at small n.
• Evaluation limited to a small DriftDiT-Small model on CIFAR-10 with three seeds.

Experimental setup

Datasets: CIFAR-10 CIFAR-100

Benchmarks: CIFAR-10 CIFAR-100

Metrics: FID wall-clock time per epoch peak GPU memory bias norm

Baselines: Standard minibatch centroid jackknife bootstrap correction BR-SNIS

Builds on: drifting models delta method for self-normalized estimators DriftDiT

Closest papers by meaning

• Drifting Fields are not Conservative (NeurIPS 2026 · similarity 0.87)
• Generative Modeling via Drifting (NeurIPS 2026 · similarity 0.87)
• Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.85)
• Distributional Diffusion Models with Scoring Rules (ICML 2025 · similarity 0.85)
• EVODiff: Entropy-aware Variance Optimized Diffusion Inference (NeurIPS 2025 · similarity 0.85)
• Elucidating the Exposure Bias in Diffusion Models (ICLR 2024 · similarity 0.85)