Research map / Image generation / Text-to-image diffusion
PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PosterMaker, an end-to-end SD3-based product poster generator with character-level OCR-feature text control (TextRenderNet) and a foreground-extension-detector reward for subject fidelity (SceneGenNet), trained in two stages.
Problem
Generating product posters that place a user-specified subject into a scene while accurately rendering multilingual (especially Chinese) text with layouts; existing pipelines are two-stage and text rendering for thousands of Chinese characters is inaccurate, and inpainting often extends subjects.
Approach
Use character-level visual features (per-character glyph images encoded by an OCR encoder, pooled, and concatenated with order and box positional encodings) as the control signal for a ControlNet-like SD3 branch, plus a learned foreground-extension detector used as a reward (ReFL-style) to improve subject fidelity, and a decoupled two-stage training schedule.
Key results
- PosterMaker reaches 93.36% sentence accuracy and 98.39% NED on PosterBenchmark, versus 86.14% for GlyphDraw2 and 80.75% for SD3 canny&inpaint.
- Character-level PPOCR features give 95.15% sentence accuracy vs 38.91% for PPOCR line-level features in the text-feature ablation.
- Subject fidelity feedback learning reduces foreground extension ratio from 15.05% to 11.57%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PosterMaker instead of the strongest baseline, GlyphDraw2 | PosterMaker · PosterBenchmark | sentence accuracy: 86.14% -> 93.36% | Helped |
| Use PosterMaker instead of the strongest baseline, GlyphDraw2 | PosterMaker · PosterBenchmark | NED: 96.78% -> 98.39% | Helped |
| Use PosterMaker instead of SD3 inpaint byt5 | PosterMaker · PosterBenchmark | FID: 65.45 -> 65.35 | Helped |
| Use PosterMaker instead of SD3 canny&inpaint | PosterMaker · PosterBenchmark | CLIP-T: 27.03 -> 27.04 | Helped |
| Use PosterMaker instead of SD3 inpaint byt5 | PosterMaker · PosterBenchmark | FG Ext. Ratio: 14.60% -> 11.57% | Helped |
| Add subject fidelity reward loss (ablation) | PosterMaker · PosterBenchmark | FG Ext. Ratio: 15.05% -> 11.57% | Helped |
| Add subject fidelity reward loss (ablation) | PosterMaker · PosterBenchmark | sentence accuracy: 93.11% -> 93.36% | No clear effect |
| Add character-level PPOCR visual features instead of line-level PPOCR features (ablation) | PosterMaker text rendering · PosterBenchmark text features | sentence accuracy: 38.91% -> 95.15% | Helped |
| Add character-level PPOCR visual features instead of line-level PPOCR features (ablation) | PosterMaker text rendering · PosterBenchmark text features | NED: 53.86% -> 98.75% | Helped |
| Use PPOCR character-level features instead of TrOCR character-level features (ablation) | PosterMaker text rendering · PosterBenchmark text features | sentence accuracy: 94.27% -> 95.15% | Helped |
| Use character-level PPOCR visual features instead of Canny features (ablation) | PosterMaker text rendering · PosterBenchmark text features | sentence accuracy: 81.50% -> 95.15% | Helped |
| Use character-level PPOCR visual features instead of ByT5 textual features (ablation) | PosterMaker text rendering · PosterBenchmark text features | sentence accuracy: 33.48% -> 95.15% | Helped |
Surprising findings
- Line-level OCR features often produce completely unrecognizable text, and canny control introduces stroke artifacts even though it scores well on some metrics.
- ByT5 char-level semantic features perform far worse than char-level OCR visual features, suggesting text rendering depends on structural rather than semantic features.
Limitations
- Training data is mostly Chinese posters, so generality to other languages is not established.
- Foreground extension ratio still about 11.6% after feedback learning.
- Baselines for GlyphByT5-v2 and GlyphDraw2 were reproduced by the authors since no open weights exist.
Experimental setup
Datasets: crawled e-commerce product posters (160k) 20k manually annotated foreground-extension images
Benchmarks: PosterBenchmark
Metrics: sentence accuracy normalized edit distance FID CLIP-T foreground extension ratio
Baselines: AnyText GlyphByT5-v2 GlyphDraw2 SD3 inpaint byt5 SD3 canny&inpaint SD3 inpaint AnyText
Builds on: Stable Diffusion 3 ControlNet ReFL GLIGEN AnyText
Closest papers by meaning
- GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models (AAAI 2025 · similarity 0.85)
- POSTA: A Go-to Framework for Customized Artistic Poster Generation (CVPR 2025 · similarity 0.84)
- Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers (NeurIPS 2025 · similarity 0.84)
- P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark (ICLR 2026 · similarity 0.84)
- DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models (CVPR 2025 · similarity 0.82)
- VODiff: Controlling Object Visibility Order in Text-to-Image Generation (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.