Research map / Image generation / Text-to-image diffusion
One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes One-Prompt-One-Story, a training-free method that concatenates all frame prompts into a single SDXL prompt and uses singular-value reweighting plus cross-attention identity consistency to generate consistent characters across frames.
Problem
Consistent subject/identity generation across multiple frames for storytelling with text-to-image diffusion models, where existing methods need training or extra modules and often lose prompt alignment.
Approach
Concatenate the identity prompt and all frame prompts into one long prompt so the text encoder's context naturally keeps identity consistent; then use singular-value reweighting (SVR+/SVR-) to separate frame semantics and an identity-prompt cross-attention injection (IPCA) to strengthen consistency, all without training.
Key results
- Achieves CLIP-T 0.8942 and DreamSim 0.1993 among training-free methods, ranking first among training-free methods in the main table.
- IP-Adapter has higher CLIP-I (0.9429) and lower DreamSim (0.1462), but the paper attributes this to similar poses and layouts.
- In a 37-person user study, the method was preferred in 48.60% of votes versus 29.80% for StoryDiffusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace vanilla SDXL generation with One-Prompt-One-Story | SDXL · Extended ConsiStory benchmark | CLIP-I: 0.8165 -> 0.9117 | Helped |
| replace vanilla SDXL generation with One-Prompt-One-Story | SDXL · Extended ConsiStory benchmark | DreamSim: 0.5292 -> 0.1993 | Helped |
| replace vanilla SDXL generation with One-Prompt-One-Story | SDXL · Extended ConsiStory benchmark | CLIP-T: 0.9074 -> 0.8942 | Hurt |
| use One-Prompt-One-Story instead of StoryDiffusion | SDXL · Extended ConsiStory benchmark | CLIP-T: 0.8877 -> 0.8942 | Helped |
| use One-Prompt-One-Story instead of IP-Adapter | SDXL · Extended ConsiStory benchmark | CLIP-I: 0.9429 -> 0.9117 | Hurt |
| use One-Prompt-One-Story instead of IP-Adapter | SDXL · Extended ConsiStory benchmark | DreamSim: 0.1462 -> 0.1993 | Hurt |
| add SVR+ to PCon (ablation) | One-Prompt-One-Story · Extended ConsiStory benchmark ablation | CLIP-T | Helped |
| add SVR- to PCon (ablation) | One-Prompt-One-Story · Extended ConsiStory benchmark ablation | CLIP-T | Helped |
| add both SVR+ and SVR- rather than either alone (ablation) | One-Prompt-One-Story · Extended ConsiStory benchmark ablation | CLIP-T: 0.8774, 0.8910 -> 0.8989 | Helped |
| add both SVR+ and SVR- rather than either alone (ablation) | One-Prompt-One-Story · Extended ConsiStory benchmark ablation | DreamSim: 0.2560, 0.2605 -> 0.2538 | Mixed |
| add IPCA after SVR+ and SVR- (ablation) | One-Prompt-One-Story · Extended ConsiStory benchmark ablation | CLIP-I: 0.8849 -> 0.9117 | Helped |
| add IPCA after SVR+ and SVR- (ablation) | One-Prompt-One-Story · Extended ConsiStory benchmark ablation | DreamSim: 0.2538 -> 0.1993 | Helped |
Surprising findings
- IP-Adapter's high identity similarity is attributed to generating similar poses and layouts rather than true identity preservation.
Limitations
- All prompts must be known in advance.
- Input length is limited by the text encoder's maximum capacity; sliding window may cause identity drift.
- Identity prompt's subjects appear in every generated image when multiple subjects are specified.
Experimental setup
Datasets: ConsiStory benchmark (extended)
Benchmarks: ConsiStory-extended benchmark (200 prompt sets)
Metrics: CLIP-T CLIP-I DreamSim VQAScore DSG FID user preference
Baselines: BLIP-Diffusion Textual Inversion The Chosen One PhotoMaker IP-Adapter ConsiStory StoryDiffusion Vanilla SDXL
Builds on: SDXL ConsiStory Free-U CarveKit
Closest papers by meaning
- Storybooth: Training-Free Multi-Subject Consistency for Improved Visual Storytelling (ICLR 2025 · similarity 0.91)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.89)
- Consistent Story Generation: Unlocking the Potential of Zigzag Sampling (NeurIPS 2025 · similarity 0.88)
- Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models (AAAI 2025 · similarity 0.88)
- Intelligent Grimm - Open-ended Visual Storytelling via Latent Diffusion Models (CVPR 2024 · similarity 0.88)
- Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.