Research map / Vision-language and multimodal / Human and embodied modeling
StarVector: Generating Scalable Vector Graphics Code from Images and Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces StarVector, a multimodal LLM (1B and 8B) that generates SVG code from images or text, trained on a 2M-sample SVG-Stack dataset and evaluated on the SVG-Bench benchmark with DinoScore.
Problem
Image vectorization and text-to-SVG generation: prior methods rely on curve/path approximation, lack semantic understanding, underuse SVG primitives, and pixel metrics like MSE misjudge SVG quality.
Approach
Treat image vectorization as inverse rendering and code generation: an MLLM (CLIP/SigLIP ViT encoder + StarCoder LLM) maps image or text tokens directly to SVG code, using primitives like rect, ellipse, polygon and text, and evaluates with a DINOv2-based perceptual DinoScore.
Key results
- StarVector-8B achieves the highest DinoScore on six of eight Image-to-SVG benchmarks (e.g., 0.966 on SVG-Stack), while LIVE leads on MSE and SSIM.
- StarVector outputs average about 3k tokens, close to ground-truth length, versus 18k for LIVE and 4.5k-20k for VTracer.
- On Text-to-SVG, StarVector-8B reaches FID 10.067 and CLIP 27.366 on SVG-FIGR, beating IconShop and GPT-4 baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scale from StarVector-1B to StarVector-8B | StarVector · SVG-FIGR | FID: 15.263 -> 10.067 | Helped |
| scale from StarVector-1B to StarVector-8B | StarVector · SVG-Stack | CLIP Score: 29.372 -> 31.307 | Helped |
| use StarVector-8B instead of StarVector-1B | StarVector · SVG-Emoji | DinoScore: 0.975 -> 0.943 | Hurt |
| use StarVector-8B instead of LIVE | Image vectorization · SVG-Diagrams | DinoScore: 0.870 -> 0.959 | Helped |
| use StarVector-8B instead of LIVE | Image vectorization · SVG-Diagrams | MSE: 0.010 -> 0.027 | Hurt |
| add SVG-specific data augmentations (rotation, color, and curve noise) (ablation) | StarVector · SVG-Emoji and SVG-Icons | performance | Helped |
| add SVG-Stack pre-training (ablation) | StarVector · downstream datasets, especially SVG-Emoji | performance | Helped |
| use CLIP rather than VQGAN or ConvNext as image encoder (ablation) | StarVector-1B · SVG-Emoji | image vectorization performance | Helped |
| use nucleus sampling with T=0.5 instead of greedy decoding (ablation) | StarVector-1B · SVG-Emoji | LPIPS: 0.251 -> 0.202 | Helped |
| use nucleus sampling with T=0.5 instead of greedy decoding (ablation) | StarVector-1B · SVG-Fonts | LPIPS: 0.019 -> 0.013 | Helped |
| use nucleus sampling with T=0.5 instead of greedy decoding (ablation) | StarVector-1B · SVG-Icons | LPIPS: 0.059 -> 0.043 | Helped |
| use nucleus sampling with T=0.5 instead of greedy decoding (ablation) | StarVector-1B · SVG-Stack | LPIPS: 0.157 -> 0.153 | Helped |
Surprising findings
- MSE often misranks visual quality; methods with lowest MSE generate bloated, artifact-heavy SVGs, while human preference favored StarVector.
- Spearman correlation between MSE and human preference is weak (0.06 and -0.10), whereas DinoScore correlates more strongly (about 0.62-0.76).
Limitations
- Constrained by 16k token context, inadequate for complex SVGs.
- Relies mainly on code prediction with minimal visual feedback, and generation is slow due to the LLM.
- Fixed image resolution (224/384) limits capture of complex diagrams; struggles with non-standard SVGs.
Experimental setup
Datasets: SVG-Stack SVG-Fonts SVG-Emoji SVG-Icons SVG-FIGR SVG-Diagrams TheStack
Benchmarks: SVG-Bench SVG-Stack SVG-Fonts SVG-Icons SVG-Emoji SVG-Diagrams SVG-FIGR
Metrics: DinoScore LPIPS SSIM MSE token length FID FID-CLIP CLIP Score
Baselines: Potrace VTracer AutoTrace Im2Vec LIVE DiffVG DeepSVG GPT-4V IconShop CodeLlama-70b GPT-4
Builds on: StarCoder CLIP LIVE DeepSVG IconShop
Closest papers by meaning
- Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models (CVPR 2025 · similarity 0.91)
- SVGBuilder: Component-Based Colored SVG Generation with Text-Guided Autoregressive Transformers (AAAI 2025 · similarity 0.91)
- SVGDreamer: Text Guided SVG Generation with Diffusion Model (CVPR 2024 · similarity 0.89)
- OmniSVG: A Unified Scalable Vector Graphics Generation Model (NeurIPS 2025 · similarity 0.89)
- VFIG: Vectorizing Complex Figures in SVG with Vision-Language Models (NeurIPS 2026 · similarity 0.87)
- Rendering-Aware Reinforcement Learning for Vector Graphics Generation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.