Research map / Image generation / Autoregressive and latent generation
Diffusion Model as Representation Learner
Xingyi Yang, Xinchao Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill from a pre-trained diffusion model (RepFusion) instead of no distillation | BiSeNetv1 ResNet-18 · CelebAMask-HQ segmentation | mIOU: 61.22 -> 67.37 | Improved |
| distill from DPM instead of a supervised ResNet-50 teacher | ResNet-18 student · CelebAMask-HQ segmentation | mIOU: 63.12 -> 67.37 | Improved |
| distill from DPM instead of ResNet-50 teacher | ResNet-50 student · CelebAMask-HQ segmentation | mIOU: 68.21 -> 70.63 | Improved |
| use RepFusion distillation instead of self-supervised pretraining (SwAV) | ResNet-18 · CelebAMask-HQ segmentation | mIOU: 67.14 -> 67.37 | No clear effect |
| add RepFusion initialization for keypoint detection | HRNetv2-w18 · WFLW | NME test: 4.33 -> 4.23 | Improved |
| add RepFusion initialization for keypoint detection | ResNet-50 · WFLW pose subset | NME pose: 8.15 -> 7.97 | Improved |
| distill from DPM on CIFAR-10 | ResNet-18 student · CIFAR-10 | average accuracy improvement: +0.42 | Improved |
| distill from DPM on Tiny-ImageNet | ResNet-18 student · Tiny-ImageNet | average accuracy improvement: +1.83 | Improved |
| use DPM distillation for classification rather than segmentation | RepFusion · CIFAR-10 vs CelebAMask-HQ | gain size: classification +0.42 vs segmentation mIOU +6.15 | Mixed |
| use random or large fixed timesteps for distillation instead of reinforced per-sample selection (ablation) | RepFusion · CIFAR-10 ResNet-18 | accuracy | Hurt |
| add reinforced per-sample time selection instead of fixed small t (ablation) | RepFusion · CIFAR-10 ResNet-18 | accuracy | Improved |
| use hint-based loss instead of AT or RKD for feature distillation (ablation) | RepFusion · CIFAR-10 and Tiny-ImageNet | accuracy | Improved |
Closest papers by meaning
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.89)
- Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think (ICLR 2025 · similarity 0.89)
- DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture (CVPR 2025 · similarity 0.89)
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.88)
- Why DiT Models Underperform as Representation Learners without Long Skip Connections (NeurIPS 2026 · similarity 0.88)
- Deconstructing Denoising Diffusion Models for Self-Supervised Learning (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
