Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
D2R2: Diffusion-based Representation with Random Distance Matching for Tabular Few-shot Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D2R2, which trains a conditional diffusion model as a representation extractor with an added random distance matching loss, then classifies tabular few-shot tasks with an instance-wise iterative prototype classifier.
Problem
Few-shot classification on tabular data, where scarce labels, heterogeneous numerical and categorical features, and unreliable prototypes make standard image/NLP few-shot methods hard to apply.
Approach
Repurpose a diffusion model as a representation learner by conditioning the noise predictor on a low-dimensional embedding of the clean input (an information bottleneck), and add a random-projection distance matching loss with separate uniform and Bernoulli projections for numerical and categorical features; classify with soft k-means style instance-wise prototypes refined over query samples.
Key results
- In 5-shot on optdigits, D2R2 reaches 90.73% accuracy versus 88.42% for STUNT (best baseline).
- In 1-shot on income, D2R2 reaches 75.82% versus 63.52% for STUNT; the paper reports 72.08% for karkunen 1-shot against STUNT's 71.20%.
- Ablation on optdigits 1-shot shows the full D2R2 at 81.13% versus 77.41% for diffusion representation plus RDM without instance-wise prototypes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use D2R2 instead of the strongest baseline | Tabular few-shot classification · cmc, 1-shot | accuracy: 37.10 -> 42.88 | Helped |
| Use D2R2 instead of the strongest baseline | Tabular few-shot classification · income, 1-shot | accuracy: 64.02 -> 75.82 | Helped |
| Use D2R2 instead of the strongest baseline | Tabular few-shot classification · karkunen, 5-shot | accuracy: 85.45 -> 84.96 | Hurt |
| Add instance-wise iterative prototypes to DR+RDM (ablation) | D2R2 · optdigits, 1-shot | accuracy: 77.41 -> 81.13 | Helped |
| Add instance-wise iterative prototypes to DR+RDM (ablation) | D2R2 · dna, 1-shot | accuracy: 61.29 -> 68.00 | Helped |
| Add instance-wise iterative prototypes to DR+RDM (ablation) | D2R2 · cmc, 1-shot | accuracy: 38.69 -> 42.88 | Helped |
| Replace center-specific prototypes with instance-wise iterative prototypes (ablation) | D2R2 · optdigits, 5-shot | accuracy: 87.12 -> 90.73 | Helped |
| Add diffusion representation to RDM+IP (ablation) | D2R2 · optdigits, 1-shot | accuracy: 26.66 -> 81.13 | Helped |
| Add diffusion representation to RDM+IP (ablation) | D2R2 · dna, 1-shot | accuracy: 26.03 -> 68.00 | Helped |
| Add diffusion representation to RDM+IP (ablation) | D2R2 · cmc, 1-shot | accuracy: 34.90 -> 42.88 | Helped |
| Add RDM to diffusion representation (ablation) | D2R2 · optdigits, 1-shot | accuracy: 72.38 -> 77.41 | Helped |
| Add RDM to diffusion representation (ablation) | D2R2 · dna, 1-shot | accuracy: 57.14 -> 61.29 | Helped |
Surprising findings
- Removing the diffusion representation (RDM only) drops accuracy sharply, e.g. optdigits 1-shot from 81.13% to 49.05%, while RDM+IP alone is near chance on several datasets.
Limitations
- Approach is tailored to tabular data and is not expected to transfer to CV or NLP.
- Sensitive to changes in column order, as the paper notes.
- No theoretical analysis of convergence or representation quality is provided.
Experimental setup
Datasets: optdigits mfeat-karhunen diabetes mfeat-pixel dna cmc income nomao breast
Benchmarks: optdigits mfeat-karhunen diabetes mfeat-pixel dna cmc income nomao breast
Metrics: accuracy Wilcoxon signed-rank test p-value
Baselines: CatBoost TabPFN KNN Mean Teacher ICT Pseudo-Label MPL SubTab VIME SCARF RTDL UMTRA
Builds on: DDPM STUNT soft k-means Random distance prediction (RDP) Center prototype classification
Closest papers by meaning
- TabDiff: a Mixed-type Diffusion Model for Tabular Data Generation (ICLR 2025 · similarity 0.86)
- TabR: Tabular Deep Learning Meets Nearest Neighbors (ICLR 2024 · similarity 0.86)
- Enhancing Tabular Learners with Context-Aware Semantic Embeddings (NeurIPS 2026 · similarity 0.86)
- Few-shot Learner Parameterization by Diffusion Time-steps (CVPR 2024 · similarity 0.86)
- Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation (NeurIPS 2024 · similarity 0.86)
- $\mathbf{D^{3}S^{2}}$: Diffusion-Guided Dataset Distillation for Semantic Segmentation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.