Research map / Label-efficient and robust learning / Self-supervised and contrastive
Distribution Augmentation for Generative Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DistAug (rotation) to 58M Sparse Transformer | Sparse Transformer 58M · CIFAR-10 test | BPD: 2.80 -> 2.62 | Helped |
| Scale DistAug strength with model size instead of reducing parameters (dropout lowered as augmentation increases) | Sparse Transformer 152M · CIFAR-10 test | BPD: 2.79 (tuned dropout + hflip) -> 2.53 (rot,tr,col,js, dropout 5%) | Helped |
| Add rotation, transposition, color, jigsaw DistAug augmentations (ablation) | Sparse Transformer 152M · CIFAR-10 validation | BPD: 2.88 (no aug) -> 2.53 | Helped |
| Add DistAug rotation to ImageNet-64 model | 152M and 303M models · ImageNet-64 validation | BPD: 3.432 -> 3.427 (152M); 3.425 -> 3.419 (303M) | Helped |
| Add DistAug rotation to 15M CIFAR-10 model (ablation) | 15M Sparse Transformer · CIFAR-10 test | BPD: 2.82 -> 2.69 | Helped |
| Use fixed random ordering as the single augmentation (ablation) | 15M Sparse Transformer · CIFAR-10 test | BPD: 2.82 -> 2.88 | Hurt |
| Use jigsaw as single augmentation (ablation) | 15M Sparse Transformer · CIFAR-10 test | BPD: 2.82 -> 2.78 | Helped |
| Use excess ImageNet data (75%) as augmentation (ablation) | 15M Sparse Transformer · CIFAR-10 test | BPD: 2.82 -> 2.80 | Helped |
| Add conditioning signal to RandAugment DistAug (99% augmentation) (ablation) | Sparse Transformer 152M · CIFAR-10 RandAugment 99% | FID: 42.0 -> 21.1 | Helped |
| Add conditioning signal to RandAugment DistAug (ablation) | Sparse Transformer 152M · CIFAR-10 RandAugment 99% | BPD: 2.715 -> 2.708 | Helped |
| Add DistAug rotation+transposition to PixelCNN++ | PixelCNN++ 53M · CIFAR-10 test | BPD: 2.93 -> 2.88 (rot,tr) | Helped |
| Add DistAug rotation+transposition to Flow++ | Flow++ 32M · CIFAR-10 test | BPD: 3.08 -> 2.98 | Helped |
Closest papers by meaning
- Differentiable Augmentation for Data-Efficient GAN Training (NeurIPS 2020 · similarity 0.90)
- Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distribution (ICML 2024 · similarity 0.88)
- Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models (NeurIPS 2023 · similarity 0.88)
- Augmentation-Aware Self-Supervision for Data-Efficient GAN Training (NeurIPS 2023 · similarity 0.88)
- AdaGAN: Boosting Generative Models (NeurIPS 2017 · similarity 0.88)
- Variational Annealing of GANs: A Langevin Perspective (ICML 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.