Research map / Image generation / Text-to-image diffusion
Storybooth: Training-Free Multi-Subject Consistency for Improved Visual Storytelling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StoryBooth, a training-free storyboard generator that combines MLLM chain-of-thought layout planning, region-based generation, bounded cross-frame self-attention to curb inter-character leakage, and cross-frame token merging for fine-grained consistency.
Problem
Maintaining subject consistency across storyboard images when multiple characters appear; existing cross-frame self-attention methods cause self-attention leakage between characters (e.g., a dog appearing like a duck).
Approach
Localize each character with LLM-based layout planning and region-based generation, then restrict cross-frame self-attention so each character's tokens attend only to same-character tokens (with a small dropout to global tokens), and add cross-frame token merging with positive/negative alpha to align fine details and control pose variance.
Key results
- Overall T2I 0.706 and CC 0.811, outperforming the best baselines (DB-LoRA 0.647/0.732; IP-Adapter 0.607/0.812) in averaged scores across single and multiple subjects.
- On multiple subjects, StoryBooth reaches T2I 0.633 and CC 0.846, versus 0.594 / 0.834 for IP-Adapter.
- Inference takes 8.7 s per storyboard on a single H100, about 30x faster than DreamBooth-LoRA (~312 s) and Textual Inversion (~455 s) per the table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Storybooth instead of StoryDiff | Storyboard generation · Single Subject | T2I: 0.788 -> 0.781 | Hurt |
| Use Storybooth instead of IP-Adapter | Storyboard generation · Single Subject | CC: 0.789 -> 0.775 | Hurt |
| Use Storybooth instead of IP-Adapter | Storyboard generation · Multiple Subjects | T2I: 0.594 -> 0.633 | Helped |
| Use Storybooth instead of IP-Adapter | Storyboard generation · Multiple Subjects | CC: 0.834 -> 0.846 | Helped |
| Use Storybooth instead of DB-LoRA | Storyboard generation · Overall | T2I: 0.647 -> 0.706 | Helped |
| Use Storybooth instead of IP-Adapter | Storyboard generation · Overall | CC: 0.812 -> 0.811 | No clear effect |
| Use Storybooth instead of StoryDiff | Storyboard generation · Storyboard generation | Inference Time: 6.5 sec -> 8.7 sec | Hurt |
| Use Storybooth instead of DB-LoRA | Storyboard generation · Storyboard generation | Inference Time: 312 sec -> 8.7 sec | Helped |
| Add self-attention bounding (ablation) | Storybooth · Multi-character storyboard generation | Inter-character leakage | Helped |
| Add cross-frame token-merging (ablation) | Storybooth · Storyboard fine-grain character features | Fine-grain consistency | Helped |
| Add early negative token unmerging (ablation) | Storybooth · Storyboard generation | Pose variance | Helped |
Surprising findings
- Naive restriction of attention to same-subject regions degrades image quality; a small random dropout allowing attention to global tokens is needed.
- Negative token merging (alpha=-0.5) in early denoising steps is used to increase pose variance, while positive merging later improves consistency.
Limitations
- Relies on LLM-based layout planning and region masks, so errors in planning propagate to generation.
- Multi-subject evaluation uses a self-constructed dataset of two randomly selected subjects, limiting generality of comparisons.
- Inference time is marginally higher than StoryDiffusion.
Experimental setup
Datasets: storyboard prompt dataset from prior work (single-subject) constructed multi-subject prompt dataset
Benchmarks: single-subject storyboard prompts constructed multi-subject dataset
Metrics: VQAScore (T2I alignment) DreamSim cosine similarity (character consistency) inference time human user study win/tie/lose
Baselines: DB-LoRA Textual Inversion IP-Adapter BLIP-Diffusion StoryGen StoryDiffusion ConsiStory
Builds on: cross-frame self-attention (StoryDiffusion) ConsiStory token merging (ToMe) region-based planning and generation
Closest papers by meaning
- One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt (ICLR 2025 · similarity 0.91)
- Consistent Story Generation: Unlocking the Potential of Zigzag Sampling (NeurIPS 2025 · similarity 0.89)
- SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistency (NeurIPS 2025 · similarity 0.89)
- Storynizor: Consistent Story Generation via Inter-Frame Synchronized and Shuffled ID Injection (AAAI 2025 · similarity 0.89)
- StoryGPT-V: Large Language Models as Consistent Story Visualizers (CVPR 2025 · similarity 0.89)
- StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.