Research map / Vision-language and multimodal / Human and embodied modeling
AESOP: Abstract Encoding of Stories, Objects, and Pictures
Hareesh Ravi, Kushal Kafle, Scott Cohen, Jonathan Brandt, Mubbasir Kapadia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text modality (compare proposed multimodal model to unimodal visual model) (ablation) | AESOP model · AESOP Assistant Illustrator | Scene similarity: 4.2 -> 4.1 | No clear effect |
| add story context (compare to one-to-one model without story context) (ablation) | AESOP model · AESOP Assistant Illustrator | Scene similarity: 1.2 -> 4.1 | Improved |
| use abstract object representation instead of pretrained ResNet-18 pixel model (ablation) | AESOP model · AESOP Assistant Illustrator | Scene similarity: 1.0 -> 4.1 | Improved |
| add visual input to writer (multimodal vs GPT-2 unimodal) (ablation) | AESOP model · AESOP Assistant Writer | BLEU-4: 0.41 -> 1.96 | Improved |
| add visual input to writer (multimodal vs GPT-2 unimodal) (ablation) | AESOP model · AESOP Assistant Writer | CIDEr: 5.7 -> 17.9 | Improved |
| add story context to writer (vs one-to-one) (ablation) | AESOP model · AESOP Assistant Writer | CIDEr: 7.2 -> 17.9 | Improved |
| compare proposed model against human baseline on writer metrics | AESOP model · AESOP Assistant Writer | METEOR: 11.1 -> 9.02 | Hurt |
| compare proposed model against human baseline on writer metrics | AESOP model · AESOP Assistant Writer | BLEU-4: 1.84 -> 1.96 | Improved |
| copy previous panel (Repeat) versus proposed model | Repeat baseline · AESOP Assistant Illustrator | Scene similarity: 5.1 -> 4.1 | Hurt |
| use AESOP creation process instead of VIST | VIST · Panel-swap human test | identification accuracy: 65.8% -> 95% | Improved |
Closest papers by meaning
- ViStoryBench: Comprehensive Benchmark Suite for Story Visualization (CVPR 2026 · similarity 0.87)
- LogiStory: A Logic-Aware Framework for Multi-Image Story Visualization (ICLR 2026 · similarity 0.87)
- Intelligent Grimm - Open-ended Visual Storytelling via Latent Diffusion Models (CVPR 2024 · similarity 0.86)
- VinaBench: Benchmark for Faithful and Consistent Visual Narratives (CVPR 2025 · similarity 0.86)
- StoryGPT-V: Large Language Models as Consistent Story Visualizers (CVPR 2025 · similarity 0.86)
- ContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Context (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
