Research map / Probabilistic and scientific modeling / Variational and generative sampling
DiScoFormer: Plug-In Density and Score Estimation with Transformers
Vasily Ilin, Peter Sushko, Ranjay Krishna
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add whitening layer (ablation) | DiScoFormer · d=1 GMM, OOD scales | score MSE: 1.136 -> 0.020 | Improved |
| add whitening layer (ablation) | DiScoFormer · d=1 GMM, OOD scales | log-density MSE: 1.593 -> 0.123 | Improved |
| add whitening layer (ablation) | DiScoFormer · d=1 GMM, in-distribution | score MSE: 0.118 -> 0.107 | Improved |
| add whitening layer (ablation) | DiScoFormer · d=1 GMM, in-distribution | log-density MSE: 0.066 -> 0.058 | Improved |
| add test-time training with consistency loss (8 steps) | DiScoFormer · 2D Student-t, n=128 | score MSE: 0.1980 -> 0.1450 | Improved |
| add test-time training (6 steps) | DiScoFormer · 2D Student-t, n=512 | score MSE: 0.0574 -> 0.0485 | Improved |
| add more TTT steps (6 to 8) | DiScoFormer · 2D Student-t, n=512 | score MSE: 0.0485 -> 0.0514 | Hurt |
| replace Scott KDE with DiScoFormer | score estimator · 2D Laplace, n=512 | score MSE: 0.3810 -> 0.3598 | Improved |
| replace Scott KDE with DiScoFormer (no TTT) | score estimator · 2D Student-t, n=128 | score MSE: 0.1515 -> 0.1980 | Hurt |
| replace Scott KDE with DiScoFormer | score estimator · GMM d=10, n=2^8 | relative score MSE (%): 65.6 -> 7.49 | Improved |
| replace best KDE with DiScoFormer | score/log-density estimator · GMM d=100 | score MSE: 1.090 -> 0.167 | Improved |
| scale context size beyond training range (2^14 to 2^17) | DiScoFormer · GMM d=10 extrapolation | relative score MSE (%): 2.83 -> 2.74 | Improved |
Closest papers by meaning
- SD-KDE: Score-Debiased Kernel Density Estimation (NeurIPS 2025 · similarity 0.87)
- Density Ratio Estimation with Conditional Probability Paths (ICML 2025 · similarity 0.86)
- ConjNorm: Tractable Density Estimation for Out-of-Distribution Detection (ICLR 2024 · similarity 0.85)
- Nonparametric Estimation of a Factorizable Density using Diffusion Models (NeurIPS 2026 · similarity 0.85)
- Sequential Neural Score Estimation: Likelihood-Free Inference with Conditional Score Based Diffusion Models (ICML 2024 · similarity 0.85)
- Sparse Attention as Compact Kernel Regression (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
