AI Paper Atlas

Research map / Image generation / Autoregressive and latent generation

Sequential Latent Spaces for Modeling the Intention During Diverse Image Captioning

Jyoti Aneja, Harsh Agrawal, Dhruv Batra, Alexander Schwing

ICCV 2019 · Image generation › Autoregressive and latent generation · approach: Latent diffusion with learned tokenizers · 60 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add ELMo-based backward LSTM in the encoder (ablation)Seq-CVAE · MSCOCO ablation (100 samples, latent 512)CIDEr (C@100): 1.089 -> 1.568Improved
use a per-word sequential latent instead of a single latent variable (ablation)Seq-CVAE with ELMo · MSCOCO ablation (20 samples)CIDEr: 0.952 -> 1.448Improved
learn the intention model (LSTM) instead of a constant Gaussian (ablation)Seq-CVAE with ELMo · MSCOCO ablation (20 samples)CIDEr: 1.244 -> 1.448Improved
use separate LSTMs for encoder, decoder and transition network (ablation)Seq-CVAE · MSCOCO ablation (20 samples)CIDEr: 1.14 -> 1.448Improved
replace pretrained ELMo with a backward RNN trained on COCO (ablation)Seq-CVAE · MSCOCO ablation (20 samples)CIDEr: 1.448 -> 1.348No clear effect
condition the intention model on previous words x_<t (ablation)Seq-CVAE with ELMo · MSCOCO ablation (20 samples, latent 512)CIDEr (C@20): 1.332 -> 1.332No clear effect
condition the intention model on previous words x_<t (ablation)Seq-CVAE with ELMo · MSCOCO ablation (100 samples, latent 256)CIDEr (C@100): 1.559 -> 1.575Improved
learned LSTM intention instead of constant Gaussian, without ELMo (ablation)Seq-CVAE without ELMo · MSCOCO ablation (100 samples, latent 512)CIDEr (C@100): 1.082 -> 1.089No clear effect
reduce latent size from 512 to 128 (ablation)Seq-CVAE with ELMo · MSCOCO ablation (100 samples)CIDEr (C@100): 1.573 -> 1.571No clear effect
use Seq-CVAE instead of POS for diversitySeq-CVAE vs POS · MSCOCO M-RNN test (20 samples, best-5)Div-2: 0.35 -> 0.54Improved
use Seq-CVAE instead of POS for novel sentencesSeq-CVAE vs POS · MSCOCO M-RNN test (100 samples, best-5)# novel sentences: 3446 -> 4215Improved
sample from Seq-CVAE instead of beam search (20 samples)Seq-CVAE vs beam search · MSCOCO M-RNN test (20 samples, best-1 oracle)CIDEr: 1.595 -> 1.448Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy