Research map / Image generation / Autoregressive and latent generation
Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Chat2SVG generates SVG templates from basic primitives with an LLM (claude-3-5-sonnet) and refines them via dual-stage latent and point optimization guided by SDEdit/ControlNet image targets.
Problem
Text-to-SVG generation struggles with shape regularity, semantic clarity of paths, generalization, and expressiveness; image-based methods fragment shapes and language-based methods lack general-domain data and geometric detail.
Approach
Combine an LLM that produces semantically labeled SVG templates from basic primitives (with prompt expansion and visual rectification) with diffusion-guided dual-stage optimization (path-level SVG VAE latent optimization, then control-point refinement) to add detail while keeping one semantic element per path.
Key results
- Chat2SVG achieves image FID 33.31 vs. 35.48 for the next best (SVGDreamer), path FID 39.07 and CLIP text alignment 0.3090 (highest).
- Removing SVG template generation raises image FID to 47.21 and path FID to 70.13.
- User study with 31 participants shows Chat2SVG results preferred over baselines on aesthetics, path regularity, and text alignment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Chat2SVG instead of SVGDreamer | Text-to-SVG generation · Text-to-SVG evaluation | Image FID: 35.48 -> 33.31 | Helped |
| use Chat2SVG instead of T2V-NPR | Text-to-SVG generation · Text-to-SVG evaluation | Path FID: 42.03 -> 39.07 | Helped |
| use Chat2SVG instead of T2V-NPR | Text-to-SVG generation · Text-to-SVG evaluation | Text Alignment: 0.3078 -> 0.3090 | Helped |
| add LLM-generated SVG templates instead of using randomly initialized shapes (ablation) | Chat2SVG · Text-to-SVG evaluation | Image FID: 47.21 -> 33.31 | Helped |
| add LLM-generated SVG templates instead of using randomly initialized shapes (ablation) | Chat2SVG · Text-to-SVG evaluation | Path FID: 70.13 -> 39.07 | Helped |
| add LLM-generated SVG templates instead of using randomly initialized shapes (ablation) | Chat2SVG · Text-to-SVG evaluation | Text Alignment: 0.2968 -> 0.3090 | Helped |
| add dual-stage SVG optimization (ablation) | Chat2SVG · Text-to-SVG evaluation | Image FID: 33.45 -> 33.31 | Helped |
| add dual-stage SVG optimization (ablation) | Chat2SVG · Text-to-SVG evaluation | Path FID: 36.12 -> 39.07 | Hurt |
| add dual-stage SVG optimization (ablation) | Chat2SVG · Text-to-SVG evaluation | Text Alignment: 0.3044 -> 0.3090 | Helped |
Surprising findings
- Removing dual-stage optimization yields the lowest path FID (36.12), since LLM primitives already match ground-truth path regularity, though visual detail suffers.
Limitations
- LLM may produce imprecise layouts and miss visual elements.
- SAM may overlook decorative regions.
- Pretrained SVG VAE with fixed command count limits intricate shape representation.
Experimental setup
Datasets: SVGRepo (52,805 colored SVG files, for FID reference) FIGR-8-SVG (path autoencoder evaluation) LLM-generated prompt set of 125 prompts
Benchmarks: SVGRepo reference FID FIGR-8-SVG path FID CLIP text alignment on 125-prompt set
Metrics: image FID path FID CLIP score user selection ratio
Baselines: CLIPDraw DiffSketcher VectorFusion SVGDreamer T2V-NPR
Builds on: T2V-NPR SDEdit ControlNet DeepSVG ImageReward
Closest papers by meaning
- SVGDreamer: Text Guided SVG Generation with Diffusion Model (CVPR 2024 · similarity 0.92)
- OmniSVG: A Unified Scalable Vector Graphics Generation Model (NeurIPS 2025 · similarity 0.91)
- StarVector: Generating Scalable Vector Graphics Code from Images and Text (AAAI 2025 · similarity 0.91)
- SVGBuilder: Component-Based Colored SVG Generation with Text-Guided Autoregressive Transformers (AAAI 2025 · similarity 0.91)
- Empowering LLMs to Understand and Generate Complex Vector Graphics (CVPR 2025 · similarity 0.90)
- InternSVG: Towards Unified SVG Tasks with Multimodal Large Language Models (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.