Research map / Learning theory and algorithms / Kernels and regression theory
Debiased Distribution Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes debiased distribution compression methods (Stein Kernel Thinning, low-rank SKT, Stein Recombination, Stein Cholesky) that turn biased MCMC/tempered/approximate-MCMC inputs into compact coresets with better-than-iid MMD.
Problem
Compressing a target distribution P into a small summary when the available input points are biased (target Q ≠ P), e.g. from burn-in, approximate MCMC, or tempering, where existing high-quality compression requires unbiased or fast-mixing inputs.
Approach
Debias the input via simplex reweighting using a mean-zero Stein kernel (greedy Stein thinning or a new low-rank adaptive debiasing), then compress the debiased approximation with kernel thinning, recombination, or Cholesky-based constant-preserving compression, achieving better-than-iid MMD even from biased inputs.
Key results
- Stein Kernel Thinning returns sqrt(n) equal-weighted points with Õ(n^{-1/2}) MMD in O(n^2) time, versus Stein thinning needing Ω(n) points for the same error.
- Low-rank SKT achieves the same guarantee in Õ(n^{1.5}) time using adaptive low-rank debiasing.
- Stein Recombination and Stein Cholesky match SKT guarantees with poly-log(n) weighted points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use proposed debiased compression methods instead of standard thinning | Biased approximate-MCMC samples for Bayesian logistic regression · Forest Covtype posterior | energy distance | Helped |
| Use proposed debiased compression methods instead of Stein thinning | Biased approximate-MCMC samples for Bayesian logistic regression · Forest Covtype posterior | MMD | Helped |
| Use proposed debiased compression methods instead of Stein thinning | Biased burn-in samples · Goodwin model posterior | MMD | Helped |
| Use proposed debiased compression methods instead of standard thinning | Biased burn-in samples · Goodwin model posterior | MMD | Helped |
| Use proposed debiased compression methods instead of Stein thinning | Biased tempered samples · Cardiac calcium signaling posterior | MMD | No clear effect |
| Use proposed weighted compression methods instead of Stein thinning | Biased tempered samples · Cardiac calcium signaling posterior | MMD | Helped |
| Use proposed debiased compression methods instead of standard thinning | Biased tempered samples · Cardiac calcium signaling posterior | MMD | Helped |
Limitations
- Guarantees for Stein kernel MMD rely on kernel growth assumptions (log-growth or poly-growth covering numbers) and slow-growing input points.
- Quadratic-time greedy variants remain O(n^2); low-rank variants need rank and step-count conditions.
Experimental setup
Datasets: Goodwin model MCMC chain (P-MALA) Bayesian logistic regression on Forest Covtype (approximate MCMC via SGFS) cardiac calcium signaling model (tempered random walk MH)
Metrics: maximum mean discrepancy energy distance integration error for mean
Baselines: Stein thinning standard thinning burn-in oracle (6-chain) importance sampling
Builds on: kernel thinning Stein thinning Compress++ randomly pivoted Cholesky recombination
Closest papers by meaning
- Supervised Kernel Thinning (NeurIPS 2024 · similarity 0.89)
- Conditional Distribution Compression via the Kernel Conditional Mean Embedding (NeurIPS 2025 · similarity 0.89)
- Low-Rank Thinning (ICML 2025 · similarity 0.87)
- LowRankArena: A Standardized Evaluation Platform for SVD-Based LLM Compression (NeurIPS 2026 · similarity 0.84)
- The Polynomial Stein Discrepancy for Assessing Moment Convergence (ICML 2025 · similarity 0.84)
- Accelerated Methods with Compressed Communications for Distributed Optimization Problems Under Data Similarity (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.