Research map / Image generation / GANs
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Bo Dai, Sanja Fidler, Raquel Urtasun, Dahua Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · MSCOCO retrieval, 5000 images (E-GAN similarity) | Recall@1: 5.06 -> 14.30 | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · MSCOCO retrieval, 5000 images (log-likelihood) | Recall@1: 9.88 -> 12.04 | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · MSCOCO retrieval, 5000 images (log-likelihood) | Recall@10: 39.94 -> 41.78 | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · MSCOCO | CIDEr: 1.020 -> 0.795 | Hurt |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · Flickr30k | CIDEr: 0.767 -> 0.202 | Hurt |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · MSCOCO | E-NGAN: 0.464 -> 0.528 | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · MSCOCO | E-GAN: 0.427 -> 0.602 | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · Flickr30k | E-GAN: 0.439 -> 0.456 | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · User study pairwise preference | fraction preferred by humans: 61% | Improved |
| add conditional GAN training (G-GAN) instead of MLE | LSTM caption generator · User study vs human descriptions | fraction won against human: 9% -> over 24% | Improved |
| use E-GAN evaluator instead of CIDEr or SPICE as metric | caption evaluation · User study agreement | Kendall rank correlation with human evaluation: -0.30 -> 0.14 | Improved |
| add adversarial training of the evaluator (E-GAN vs E-NGAN) (ablation) | caption evaluator · Human vs generator score gap | discriminative power | Improved |
Closest papers by meaning
- Improving Image Captioning with Conditional Generative Adversarial Nets (AAAI 2019 · similarity 0.92)
- Adversarial Learning of Semantic Relevance in Text to Image Synthesis (AAAI 2019 · similarity 0.91)
- Adversarial Semantic Alignment for Improved Image Captions (CVPR 2019 · similarity 0.91)
- Diversity-Sensitive Conditional Generative Adversarial Networks (ICLR 2019 · similarity 0.90)
- Linear Semantics in Generative Adversarial Networks (CVPR 2021 · similarity 0.89)
- Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
