Research map / Image generation / GANs
InstantPainting: Expanding GANs for Efficient Text-Conditioned Image Generation Platform
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents InstantPainting, a web/mobile text-to-image platform built on an extended GALIP GAN that claims about 3% of other platforms' pre-training data and 4-5x faster generation.
Problem
Large-scale diffusion-based text-to-image platforms are expensive to train and deploy and slow to generate images.
Approach
Expands the GALIP GAN with an added convolutional discriminator module, a noise selection module in the generator, and a ViT-16 CLIP encoder, trained on LAION-400M plus a curated ~1M-pair fine-tuning set in a multi-stage scheme, and wraps it in a Flask/SQLite PC and mobile web application.
Key results
- Claims about 3% of other platforms' pre-training data and 4% of parameters (0.23B vs. 3-6.6B for baselines).
- Claims approximately 4 to 5 times faster generation speed than existing platforms.
- Deployment cost reported as 16 RTX A6000 versus at least 256 Tesla V100 for SD-XL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an expanded GAN platform instead of platforms based on large-scale diffusion models | InstantPainting · Text-conditioned image generation | generation speed: approximately 4 to 5 times faster | Helped |
| Use an expanded GAN platform with about 3% of other platforms’ pre-training data | InstantPainting · Platform comparison | pre-training data: about 3% | Helped |
| Use an expanded GAN platform with only 4% of the parameters | InstantPainting · Platform comparison | parameters: 4% | Helped |
| Use the expanded GAN platform | InstantPainting · Platform comparison | deployment cost | Helped |
Limitations
- No quantitative image quality evaluation or speed benchmark is reported in the demo paper.
- Comparisons to baselines in Table 1 rely on reported figures rather than controlled experiments.
Experimental setup
Datasets: LAION-400M curated fine-tuning set (~1M pairs)
Baselines: SD-XL RAPHAEL DALL-E 3
Builds on: GALIP
Closest papers by meaning
- TiGAN: Text-Based Interactive Image Generation and Manipulation (AAAI 2022 · similarity 0.87)
- Constrained Generative Adversarial Networks for Interactive Image Generation (CVPR 2019 · similarity 0.87)
- E$^2$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation (ICML 2024 · similarity 0.87)
- Adversarial Text to Continuous Image Generation (CVPR 2024 · similarity 0.86)
- UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs (CVPR 2024 · similarity 0.85)
- Diffusion-driven GAN Inversion for Multi-Modal Face Image Generation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.