Research map / Label-efficient and robust learning / Self-supervised and contrastive
Return of Unconditional Generation: A Self-supervised Representation Generation Method
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Representation-Conditioned Generation (RCG), which unconditionally generates MoCo v3 self-supervised representations with a lightweight diffusion model and uses them to condition an image generator, closing much of the unconditional-vs-conditional gap on ImageNet.
Problem
Unconditional image generation lags far behind class-conditional generation on complex data such as ImageNet, because human labels supply semantic information that unlabeled models lack.
Approach
Generate a compact self-supervised representation unconditionally with a representation diffusion model (RDM), then use that representation as the condition for any image generator (ADM, LDM, DiT, MAGE), so no human labels are needed; classifier-free-style guidance is also enabled via representation dropout.
Key results
- RCG (MAGE-L) with guidance (RCG-G) achieves FID 2.15 and IS 253.4 on ImageNet 256x256 unconditional generation, down from the previous best 5.91 (RDM-IN).
- RCG reduces unconditional FID by 71%, 76%, 82%, 54%, and 51% for LDM-8, ADM, DiT-XL/2, MAGE-B, and MAGE-L respectively.
- Unconditional RCG (MAGE-L, 3.44 FID) is comparable to leading class-conditional methods, and with RCG the unconditional LDM-8, ADM and DiT-XL/2 outperform their class-conditional versions in an apple-to-apple comparison.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition image generators on self-supervised generated representations | LDM-8, ADM, DiT-XL/2, MAGE-B, and MAGE-L · ImageNet 256×256 | FID: 39.13 -> 11.30; 26.21 -> 6.24; 27.32 -> 4.89; 8.67 -> 3.98; 7.04 -> 3.44 | Helped |
| add classifier-free guidance | RCG (MAGE-L) · ImageNet 256×256 | FID: 3.44 -> 2.15 | Helped |
| condition on generated representations instead of no condition (ablation) | MAGE-B · ImageNet 256×256 validation | FID: 14.23 -> 5.07 | Helped |
| use MoCo v3 rather than DINO or iBOT representations (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 7.53 -> 5.07; 8.05 -> 5.07 | Helped |
| increase encoder size from ViT-B to ViT-L (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 5.07 -> 5.06 | Helped |
| reduce projection dimension from 256 to 32 (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 5.07 -> 9.14 | Hurt |
| increase projection dimension from 256 to 768 (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 5.07 -> 6.10 | Hurt |
| increase RDM depth from 3 to 12 blocks (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 7.53 -> 5.07 | Helped |
| reduce RDM hidden dimension from 1536 to 256 (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 5.07 -> 12.99 | Hurt |
| increase RDM training from 10 to 100 epochs (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 5.94 -> 5.07 | Helped |
| increase RDM sampling steps from 20 to 250 (ablation) | RCG with MAGE-B · ImageNet 256×256 validation | FID: 5.80 -> 5.07 | Helped |
| replace generated representations with class labels (ablation) | MAGE-B · ImageNet 256×256 validation | FID: 5.07 -> 5.83 | Hurt |
Surprising findings
- Unconditional generators conditioned on self-generated representations outperform their class-label-conditioned counterparts (e.g., DiT-XL/2 FID 4.89 vs 9.62 with class labels).
- Representations from MoCo v3 (trained with InfoNCE) outperform DINO and iBOT representations for generation, which the authors link to higher uniformity of the representation.
Limitations
- Generated images can be unrealistic or low quality, and generators struggle with text, regular shapes, and realistic humans.
- Relies on a pre-trained self-supervised encoder (MoCo v3) that was trained on ImageNet labels-free but on ImageNet training data.
Experimental setup
Datasets: ImageNet-1K CIFAR-10 iNaturalist 2021
Benchmarks: ImageNet 256x256 CIFAR-10 iNaturalist 2021
Metrics: FID Inception Score precision recall Frechet distance on representations
Baselines: ADM LDM-8 DiT-XL/2 MAGE-B MAGE-L BigGAN MaskGIT RCDM IC-GAN ADDP RDM-IN VDM++
Builds on: MoCo v3 MAGE DDIM classifier-free guidance Muse
Closest papers by meaning
- Can Generative Models Improve Self-Supervised Representation Learning? (AAAI 2025 · similarity 0.88)
- Understand Before You Generate: Self-Guided Training for Autoregressive Image Generation (NeurIPS 2025 · similarity 0.87)
- The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better (NeurIPS 2024 · similarity 0.86)
- Positive-Unlabeled Diffusion Models for Preventing Sensitive Data Generation (ICLR 2025 · similarity 0.86)
- A Simple Approach to Unifying Diffusion-based Conditional Generation (ICLR 2025 · similarity 0.85)
- Restyling Unsupervised Concept Based Interpretable Networks with Generative Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.