Research map / Image generation / Autoregressive and latent generation
VecFusion: Vector Font Generation with Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VecFusion, a two-stage cascaded diffusion model (raster diffusion with control point fields, then transformer-based vector diffusion over a mixed discrete-continuous control point tensor) that generates editable vector fonts.
Problem
Generating vector fonts with diverse topologies and precise, editable control point placement is hard for existing VAE, autoregressive, and CNN/RNN-based generative models.
Approach
A cascade of a raster diffusion model that generates a low-resolution glyph image plus control point fields, followed by a transformer vector diffusion model that denoises a mixed discrete-continuous control point tensor (path membership, grid cell index, and continuous displacement) conditioned on the raster output.
Key results
- On the full missing-glyph test set, VecFusion reaches L1 0.014 and CD 0.16 versus ChiroDiff's 0.044 and 1.66.
- On the harder unseen-family subset, VecFusion achieves L1 0.021 and CD 0.35 versus ChiroDiff's 0.072 and 3.63.
- For few-shot style transfer, VecFusion reports L1 0.069 and CD 0.46 versus DeepVecFont-v2's 0.098 and 1.05 and DualVector's 0.197 and 1.37.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VecFusion instead of ChiroDiff | Missing glyph generation · Full test set | L1: 0.044 -> 0.014 | Helped |
| Use VecFusion instead of ChiroDiff | Missing glyph generation · Challenging subset | L1: 0.072 -> 0.021 | Helped |
| Use VecFusion instead of DeepVecFont-v2 | Few-shot font style transfer · Test fonts | L1: 0.098 -> 0.069 | Helped |
| Use VecFusion instead of DualVector | Few-shot font style transfer · Test fonts | #vp diff: 0.428 -> 0.033 | Helped |
| predict absolute coordinates instead of grid cell indices plus relative displacements (ablation) | VecFusion · Missing glyph generation, full test set | L1: 0.014 -> 0.020 | Hurt |
| remove control point fields from the raster image (ablation) | VecFusion · Missing glyph generation, full test set | CD: 0.16 -> 0.60 | Hurt |
| remove the raster diffusion stage and use vector diffusion only (ablation) | VecFusion · Missing glyph generation, full test set | CD: 0.16 -> 0.68 | Hurt |
Surprising findings
- Off-the-shelf vectorization of the generated raster glyph fails to produce coherent curve topology, indicating the vector diffusion stage is needed.
Limitations
- Requires vector paths as supervision and cannot exploit raster-only images.
- Hallucinates uppercase glyphs for lowercase missing characters (e.g., dong sign on uppercase font), failing to preserve symbolic meaning.
- Each generation step uses 1000 DDPM sampling steps, making inference relatively slow (~10 s per glyph).
Experimental setup
Datasets: Google Fonts (1424 fonts, 324K glyphs, new dataset) DeepVecFont-v2 dataset (8,035 training fonts, 1,425 test fonts)
Benchmarks: Google Fonts missing glyph test split (5K glyphs) DeepVecFont-v2 few-shot style transfer test set
Metrics: L1 pixel difference bidirectional Chamfer distance control point count difference vector path count difference
Baselines: ChiroDiff DeepVecFont-v2 DualVector VecFusion vector only ablation VecFusion no control point fields ablation VecFusion continuous coordinates ablation
Builds on: DDPM cascaded diffusion models Analog Bits DeepVecFont-v2 ChiroDiff
Closest papers by meaning
- Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models (CVPR 2024 · similarity 0.87)
- FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning (AAAI 2024 · similarity 0.87)
- VecGlypher: Unified Vector Glyph Generation with Language Models (CVPR 2026 · similarity 0.85)
- Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models (CVPR 2025 · similarity 0.85)
- VQ-FONT: Few-Shot Font Generation with Structure-Aware Enhancement and Quantization (AAAI 2024 · similarity 0.83)
- MatFuse: Controllable Material Generation with Diffusion Models (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.