Research map / Image generation / Text-to-image diffusion
SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SceneDecorator, a training-free framework for scene-oriented story generation that uses VLM-guided global-to-local scene planning and long-term scene-sharing attention to keep scenes consistent across multiple stories.
Problem
Story generation with text-to-image models focuses on character consistency and overlooks scenes; it lacks scene-level narrative coherence in planning and scene consistency across multiple stories.
Approach
A VLM conceptualizes a global scene, which a T2I model renders; the VLM then crops it into local scenes and story sub-prompts. Generation uses mask-guided IP-Adapter-style scene injection, scene-sharing attention across story branches, and extrapolable noise blending to scale to many stories with fixed memory.
Key results
- SceneDecorator achieves DreamSim-I 0.605 and DINO-F 0.571, best on both, versus ConsiStory at 0.723 and 0.475.
- In a 61-response user study, SceneDecorator is preferred for scene alignment at 76.2% versus 14.1% for the second-best ConsiStory.
- Extrapolable noise blending keeps GPU memory fixed at 12.7G from 5 to 25 story images, while the variant without it reaches 23.5G at 20 images and OOM at 25.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SceneDecorator instead of ConsiStory | SDXL story generation · 2,920 scene-prompt pairs | CLIP-T: 0.320 -> 0.312 | Hurt |
| Use SceneDecorator instead of ConsiStory | SDXL story generation · 2,920 scene-prompt pairs | DreamSim-I: 0.723 -> 0.605 | Helped |
| Use SceneDecorator instead of ConsiStory | SDXL story generation · 2,920 scene-prompt pairs | DINO-F: 0.475 -> 0.571 | Helped |
| Use SceneDecorator instead of ConsiStory | SDXL story generation · 2,920 scene-prompt pairs | Text alignment: 21.3% -> 56.5% | Helped |
| Use SceneDecorator instead of ConsiStory | SDXL story generation · 2,920 scene-prompt pairs | Scene alignment: 14.1% -> 76.2% | Helped |
| Use SceneDecorator instead of ConsiStory | SDXL story generation · 2,920 scene-prompt pairs | Image quality: 24.7% -> 57.5% | Helped |
| Add Mask-Guided Scene Injection (ablation) | SceneDecorator · Scene-oriented story generation | Scene semantics and subject-style diversity | Helped |
| Add Scene-Sharing Attention (ablation) | SceneDecorator · Stories generated from different prompts | Scene consistency | Helped |
| Add Extrapolable Noise Blending (ablation) | SceneDecorator · 5 generated story images | GPU memory: 14.5G -> 12.7G | Helped |
| Add Extrapolable Noise Blending (ablation) | SceneDecorator · 10 generated story images | GPU memory: 17.5G -> 12.7G | Helped |
| Add Extrapolable Noise Blending (ablation) | SceneDecorator · 15 generated story images | GPU memory: 20.4G -> 12.7G | Helped |
| Add Extrapolable Noise Blending (ablation) | SceneDecorator · 20 generated story images | GPU memory: 23.5G -> 12.7G | Helped |
Surprising findings
- ConsiStory achieves the highest CLIP-T score (0.320) above SceneDecorator (0.312), so text alignment is not best on that metric.
Limitations
- Depends on limitations of underlying foundation models (FLUX.1, SDXL, Qwen2-VL).
- IP-Adapter performs less reliably in out-of-distribution scene injection, e.g., an elephant in the sky.
- Evaluation uses a self-generated set of 146 themes and a small user study of 61 responses.
Experimental setup
Metrics: CLIP-T DreamSim-I DINO-F user study preference GPT-4o narrative coherence score
Baselines: CustomDiffusion ConsiStory StoryDiffusion
Builds on: IP-Adapter StoryDiffusion ConsiStory DDIM
Closest papers by meaning
- Storybooth: Training-Free Multi-Subject Consistency for Improved Visual Storytelling (ICLR 2025 · similarity 0.89)
- ContextualStory: Consistent Visual Storytelling with Spatially-Enhanced and Storyline Context (AAAI 2025 · similarity 0.88)
- Consistent Story Generation: Unlocking the Potential of Zigzag Sampling (NeurIPS 2025 · similarity 0.88)
- Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models (AAAI 2025 · similarity 0.88)
- VinaBench: Benchmark for Faithful and Consistent Visual Narratives (CVPR 2025 · similarity 0.88)
- StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.