Research map / Image generation / Autoregressive and latent generation
Low-Biased General Annotated Dataset Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates a low-biased ImageNet-like annotated dataset by fine-tuning Stable Diffusion 1.5 with LoRA using only class names, guided by a CLIP-based bi-level semantic alignment loss and a Q-Align quality loss.
Problem
Manually collected general annotated datasets such as ImageNet contain systematic non-transferable bias (background, style, context), which degrades cross-category and cross-domain generalization of pre-trained backbones; existing diffusion-generated ImageNet-like datasets replicate this bias.
Approach
Fine-tune a pre-trained diffusion model with reward-style gradients so generated images align, via a GAN-style adversarial loss over the whole class-name text distribution and a per-image cosine loss to "photo of [class]" in CLIP space, plus an image-quality loss from Q-Align, using only the 1000 ImageNet class names as input.
Key results
- Pre-training ResNet50 on lbGen data gives 73.2% average transfer accuracy over eight datasets vs 71.5% for real ImageNet, and ViT-S gives 74.4% vs 72.3%.
- With 20% of downstream data, lbGen pre-training yields up to 1.54% AP box gain on COCO over real ImageNet pre-training.
- Texture inclination drops by 4.8% (ResNet50) and 9.8% (ViT-S) relative to real-image pre-training on Cue Conflict.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pre-train ResNet50 on generated low-biased ImageNet rather than IN-RealFake | ResNet50 · Eight transfer learning datasets (Avg.) | average accuracy: 71.8 -> 73.2 | Helped |
| Pre-train ViT-S on generated low-biased ImageNet rather than IN-Real | ViT-S · Eight transfer learning datasets (Avg.) | average accuracy: 72.3 -> 74.4 | Helped |
| Pre-train ResNet50 on lbGen instead of IN-Real | ResNet50 · COCO (1.0×) | AP box: 39.32 -> 39.26 | Hurt |
| Pre-train ResNet50 on lbGen instead of IN-Real | ResNet50 · COCO (0.2×) | AP box: 29.14 -> 30.68 | Helped |
| Pre-train ResNet50 on lbGen instead of IN-Real | ResNet50 · ADE20K (1.0×) | mIoU: 42.44 -> 41.50 | Hurt |
| Pre-train ResNet50 on lbGen instead of IN-RealFake | ResNet50 · ADE20K (0.2×) | mIoU: 32.28 -> 33.57 | Helped |
| Pre-train on lbGen rather than real ImageNet | ResNet50 · Cue Conflict | texture inclination (TI): 60.9 -> 56.1 | Helped |
| Pre-train on lbGen rather than real ImageNet | ResNet50 · FOCUS | CBavg.: 60.0 -> 64.7 | Helped |
| Pre-train on lbGen rather than real ImageNet | ResNet50 · Mixed-Rand & Mixed-Same | BGGap: 6.8 -> 6.4 | Helped |
| Add entire dataset semantic alignment loss (ablation) | lbGen · ImageNet-100 (Avg. transfer learning) | average accuracy: 53.6 -> 58.5 | Helped |
| Add individual image semantic alignment loss (ablation) | lbGen · ImageNet-100 (Avg. transfer learning) | average accuracy: 42.9 -> 58.5 | Helped |
| Add quality assurance loss (ablation) | lbGen · ImageNet-100 (Avg. transfer learning) | average accuracy: 53.7 -> 58.5 | Helped |
Surprising findings
- Higher ImageNet validation accuracy does not necessarily correlate with better cross-category generalization.
- Real-data pre-training is marginally better on full-data COCO and ADE20K, with lbGen winning only under limited downstream data.
Limitations
- Ablations were run only on the smaller ImageNet-100 setting due to computational cost.
- Generator quality depends on CLIP capacity; a smaller CLIP lowered transfer accuracy to 56.7%.
Experimental setup
Datasets: ImageNet-1K ImageNet-100 GenRobust RealFake
Benchmarks: Aircraft Cars196 DTD EuroSAT Flowers Pets Food101 SUN397 COCO ADE20K FOCUS Mixed-Rand
Metrics: top-1 accuracy AP box mIoU texture inclination CBavg BGGap
Baselines: IN-Real (ImageNet-1K) IN-SD1.5 IN-GenRobust IN-RealFake
Builds on: Stable Diffusion CLIP Q-Align Deep Reward CoMat fakeit
Closest papers by meaning
- Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusion (CVPR 2025 · similarity 0.86)
- Panoptic Scene Graph Generation with Semantics-Prototype Learning (AAAI 2024 · similarity 0.86)
- Understanding Bias in Large-Scale Visual Datasets (NeurIPS 2024 · similarity 0.85)
- GS-Bias: Global-Spatial Bias Learner for Single-Image Test-Time Adaptation of Vision-Language Models (ICML 2025 · similarity 0.85)
- Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers (CVPR 2025 · similarity 0.85)
- On Fairness of Unified Multimodal Large Language Model for Image Generation (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.