Research map / Image generation / Text-to-image diffusion
Storynizor: Consistent Story Generation via Inter-Frame Synchronized and Shuffled ID Injection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Storynizor is a Stable Diffusion 1.5-based model for consistent multi-character story image generation, combining an ID-Synchronizer (auto-mask self-attention plus mask perceptual loss) with an ID-Injector trained with a Shuffling Reference Strategy, and releases the 100k-image StoryDB dataset.
Problem
Generating continuous story images with consistent characters across frames, flexible poses, and clear foreground-background separation; existing ID-preserving methods lack inter-frame coherence and training-free story methods degrade semantically.
Approach
Auto-mask spatial self-attention restricts cross-frame attention to character regions derived from cross-attention maps (supervised by a Dice-based mask perceptual loss), while a shuffled reference-ID injection pairs each frame with a same-identity reference from a different pose/scene to avoid copying the reference pose.
Key results
- Prompt-only generation: Storynizor reaches CLIP-T 33.28, CLIP-I 83.33, DINO-I 86.62 vs ConsiStory's 29.01/76.24/79.22.
- Prompt-ID generation: Storynizor reaches CLIP-I 80.86 and Face Sim 39.64 vs InstantID's Face Sim 68.36 (InstantID is higher on face similarity but has much lower CLIP-T 25.44).
- Ablation: adding AMSA and MPL raises CLIP-I from 78.83 to 83.28 and DINO-I from 81.29 to 85.58 at 512x512; SRS beats stacked-ID embedding on Face Sim (36.48 vs 19.26).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Storynizor instead of the strongest baseline | Prompt-only story generation · Prompt-only story generation | CLIP-T: 30.01 -> 33.28 | Helped |
| Use Storynizor instead of the strongest baseline | Prompt-only story generation · Prompt-only story generation | CLIP-I: 76.24 -> 83.33 | Helped |
| Use Storynizor instead of the strongest baseline | Prompt-only story generation · Prompt-only story generation | DINO-I: 79.22 -> 86.62 | Helped |
| Use Storynizor instead of the strongest baseline | Prompt-only story generation · Prompt-only story generation | Face Sim: 30.84 -> 41.55 | Helped |
| Use Storynizor instead of InstantID | Prompt-ID story generation · Prompt-ID story generation | CLIP-I: 79.46 -> 80.86 | Helped |
| Use Storynizor instead of InstantID | Prompt-ID story generation · Prompt-ID story generation | Face Sim: 68.36 -> 39.64 | Hurt |
| Add AMSA (ablation) | Storynizor ID-Synchronizer · Story generation ablation | CLIP-T: 30.46 -> 32.51 | Helped |
| Add AMSA (ablation) | Storynizor ID-Synchronizer · Story generation ablation | CLIP-I: 78.83 -> 81.66 | Helped |
| Add AMSA (ablation) | Storynizor ID-Synchronizer · Story generation ablation | DINO-I: 81.29 -> 83.74 | Helped |
| Add MPL to AMSA (ablation) | Storynizor ID-Synchronizer · Story generation ablation | Face Sim: 31.90 -> 36.55 | Helped |
| Use Shuffling Reference Strategy instead of stacked ID embedding (ablation) | Storynizor ID-Injector · ID-injection ablation | CLIP-T: 30.39 -> 32.59 | Helped |
| Use Shuffling Reference Strategy instead of stacked ID embedding (ablation) | Storynizor ID-Injector · ID-injection ablation | CLIP-I: 70.74 -> 71.65 | Helped |
Surprising findings
- InstantID achieves the highest face similarity (68.36) but has the lowest CLIP-T among ID methods, which the authors attribute to outputs overly similar to the reference and lacking diversity.
Limitations
- ID-Injector only injects facial features; clothing consistency relies on the ID-Synchronizer and is not explicitly controlled.
- Multi-character generation without reference images only; regional methods must be integrated for multi-character reference inputs.
- Evaluation relies on GPT-4V-generated prompts and a 25-expert user study of limited scale; no release of StoryDB stated.
Experimental setup
Datasets: StoryDB LAION-Face FFHQ
Benchmarks: GPT-4V-generated prompt groups (10k test groups) FFHQ faces
Metrics: CLIP-T CLIP-I DINO-I Face Sim Face Sim (R) human win rate
Baselines: StoryGen ConsiStory StoryDiffusion IP-Adapter PhotoMaker InstantID
Builds on: Stable Diffusion 1.5 IP-Adapter ConsiStory StoryDiffusion Dice loss
Closest papers by meaning
- Storybooth: Training-Free Multi-Subject Consistency for Improved Visual Storytelling (ICLR 2025 · similarity 0.89)
- SceneDecorator: Towards Scene-Oriented Story Generation with Scene Planning and Scene Consistency (NeurIPS 2025 · similarity 0.88)
- One-Prompt-One-Story: Free-Lunch Consistent Text-to-Image Generation Using a Single Prompt (ICLR 2025 · similarity 0.87)
- StoryDiffusion: Consistent Self-Attention for Long-Range Image and Video Generation (NeurIPS 2024 · similarity 0.87)
- EntityBench: Towards Entity-Consistent Long-Range Multi-Shot Video Generation (NeurIPS 2026 · similarity 0.87)
- Intelligent Grimm - Open-ended Visual Storytelling via Latent Diffusion Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.