Research map / Vision-language and multimodal / Multimodal LLMs
Paper2Poster: Towards Multimodal Poster Automation from Scientific Papers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Paper2Poster, a benchmark and metric suite (including the PaperQuiz VLM-reader evaluation) for academic poster generation from papers, plus PosterAgent, a top-down multi-agent Parser–Planner–Painter-Commenter pipeline that matches GPT-4o baselines at a fraction of the token cost.
Problem
Generating a single visually coherent academic poster from a long, multimodal scientific paper is hard, and no benchmark or metric existed to evaluate poster quality on information transfer and design.
Approach
Treat poster generation as multimodal context compression: a Parser builds a text/visual asset library, a Planner matches sections to figures and lays panels out with a binary-tree layout, and a Painter–Commenter loop renders python-pptx panels and refines them with VLM visual feedback (zoom-in plus in-context references). Evaluation adds PaperQuiz, where VLM readers answer o3-generated questions
Key results
- PosterAgent-4o reaches an overall VLM-as-Judge score of 3.72, close to the GT poster (3.77) and above 4o-HTML (3.52), 4o-Image (2.33), and PPTAgent-4o (2.11).
- PosterAgent-4o uses about 101.1K tokens per paper versus 361K for OWL-4o, and PosterAgent-Qwen uses 47.6K tokens at about $0.0045 per poster.
- PosterAgent-Qwen (open-source) achieves a PaperQuiz raw overall of 57.46 vs 59.15 for GT posters and 50.18 for 4o-Image.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PosterAgent-4o instead of 4o-Image | Poster generation · Paper2Poster | Overall VLM-as-Judge: 2.33 -> 3.72 | Helped |
| Use PosterAgent-Qwen instead of 4o-HTML | Poster generation · Paper2Poster | PaperQuiz raw overall: 56.59 -> 57.46 | Helped |
| Use PosterAgent-4o instead of PPTAgent-4o | Poster generation · Paper2Poster | PaperQuiz raw overall: 31.25 -> 58.21 | Helped |
| Generate panel content in parallel rather than sequentially | PosterAgent-4o · PosterAgent runtime | Total runtime: 281.48s -> 166.80s; 40.7% | Helped |
| Replace binary-tree layout with direct LLM layout generation (ablation) | PosterAgent-4o · Paper2Poster | Layout quality | Hurt |
| Remove the commenter from the binary-tree layout system (ablation) | PosterAgent-4o · Paper2Poster | Layout quality | Hurt |
| Add a commenter to the binary-tree layout system (ablation) | PosterAgent-4o · Paper2Poster | Layout quality | Helped |
| Remove in-context examples from the commenter (ablation) | PosterAgent-4o · Paper2Poster | Layout quality | Hurt |
| Add in-context examples to the commenter (ablation) | PosterAgent-4o · Paper2Poster | Layout quality | Helped |
| Use Qwen-2.5-7B as VLM commenter instead of GPT-4o (ablation) | PosterAgent with GPT-4o LLM · Paper2Poster | PaperQuiz density-augmented overall: 116.13 -> 118.25; +2.1 | Helped |
| Use GPT-4o as VLM commenter instead of Qwen-2.5-7B (ablation) | PosterAgent with Qwen-2.5-7B LLM · Paper2Poster | PaperQuiz density-augmented overall: 114.65 -> 114.09; +0.56 over Qwen-4o | Hurt |
| Use GPT-4o as VLM commenter instead of Qwen-2.5-7B (ablation) | PosterAgent with GPT-4o LLM · Paper2Poster | Overall VLM-as-Judge: 3.58 -> 3.72 | Helped |
Surprising findings
- Author-designed GT posters have relatively high perplexity (11.26) and the highest PaperQuiz scores come from full papers without length penalty, suggesting visual rather than textual conveyance of meaning.
- Open-source PosterAgent-Qwen outperforms GPT-4o-based HTML and OWL baselines on raw PaperQuiz accuracy despite using weaker backbones.
Limitations
- Sequential panel refinement is the main efficiency bottleneck (about 4.5 minutes per document), partly addressed by a parallel variant.
- Benchmark is small (100 paper-poster pairs) and restricted to AI conference papers.
- Engagement scores remain below 3 for all methods, and a gap to human posters persists.
Experimental setup
Datasets: PosterSum
Benchmarks: Paper2Poster
Metrics: CLIP visual similarity figure relevance perplexity VLM-as-judge aesthetic and information scores PaperQuiz accuracy density-augmented score token cost runtime
Baselines: GPT-4o HTML GPT-4o Image OWL PPTAgent
Builds on: PPTAgent ReAct PosterSum Marker Docling
Closest papers by meaning
- P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark (ICLR 2026 · similarity 0.94)
- POSTA: A Go-to Framework for Customized Artistic Poster Generation (CVPR 2025 · similarity 0.87)
- GlyphDraw2: Automatic Generation of Complex Glyph Posters with Diffusion Models and Large Language Models (AAAI 2025 · similarity 0.86)
- Presenting a Paper is an Art: Self-Improvement Aesthetic Agents for Academic Presentations (ICLR 2026 · similarity 0.86)
- ScImage: How good are multimodal large language models at scientific text-to-image generation? (ICLR 2025 · similarity 0.85)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.