Research map / Image generation / Autoregressive and latent generation
VQ-FONT: Few-Shot Font Generation with Structure-Aware Enhancement and Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VQ-Font, a VQGAN-based few-shot Chinese font generator that refines synthesized glyphs with a pre-trained font codebook and reweights cross-attention with structure-level (radical/component) correspondences.
Problem
Few-shot font generation from a handful of reference glyphs often yields missing details and distorted strokes, especially for complex Chinese font styles (serif, artistic, handwriting).
Approach
Pre-train a VQGAN codebook on font images to encode stroke priors, predict codebook indices of the synthesized glyph with a Transformer to refine it, and add a structure-level attention reweighting (SSEM) that averages patch-level attention over Chinese character structure components.
Key results
- On UFUC, VQ-Font reports L1 0.0621, PSNR 20.249, SSIM 0.812, LPIPS 0.103, beating FS-Font (L1 0.0666, LPIPS 0.137).
- Relative to the second-best method, L1 improves 7.99% on SFUC and 6.76% on UFUC; LPIPS improves 13.51% and 11.21%.
- In the user study, VQ-Font was preferred for style consistency 62.9% (SFUC) and 67.4% (UFUC).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the pre-trained font codebook (ablation) | VQ-Font baseline · UFUC | SSIM: 0.793 -> 0.810 | Helped |
| add the pre-trained font codebook (ablation) | VQ-Font baseline · UFUC | LPIPS: 0.116 -> 0.105 | Helped |
| add SSEM to the codebook-enhanced model (ablation) | VQ-Font +C · UFUC | PSNR: 20.170 -> 20.249 | Helped |
| fine-tune the pre-trained decoder instead of freezing it (ablation) | VQ-Font · UFUC | L1: 0.0648 -> 0.0621 | Helped |
| fine-tune the pre-trained decoder instead of freezing it (ablation) | VQ-Font · UFUC | LPIPS: 0.121 -> 0.103 | Helped |
| increase codebook size from 128 to 512 (ablation) | VQ-Font · UFUC | L1: 0.0638 -> 0.0631 | Helped |
| increase codebook size from 512 to 1024 (ablation) | VQ-Font · UFUC | L1: 0.0631 -> 0.0621 | Helped |
| increase codebook size from 128 to 1024 (ablation) | VQ-Font · UFUC | SSIM: 0.807 -> 0.812 | Helped |
| increase codebook size from 128 to 1024 (ablation) | VQ-Font · UFUC | LPIPS: 0.108 -> 0.103 | Helped |
| use VQ-Font instead of FS-Font | few-shot font generation · SFUC | L1: 0.0663 -> 0.0610 | Helped |
| use VQ-Font instead of CF-Font | few-shot font generation · SFUC | LPIPS: 0.111 -> 0.096 | Helped |
| use VQ-Font instead of FS-Font | few-shot font generation · UFUC | L1: 0.0666 -> 0.0621 | Helped |
Limitations
- Evaluated on a single self-collected dataset of 382 fonts with a fixed Kai content font.
- Relies on hand-defined structure partitions of 12 Chinese character structure types.
Experimental setup
Datasets: collected font dataset of 382 fonts (3499 Chinese characters each)
Benchmarks: SFUC (seen fonts, unseen chars) UFUC (unseen fonts, unseen chars)
Metrics: L1 RMSE PSNR SSIM LPIPS user study
Baselines: LF-Font MX-Font DG-Font FS-Font CF-Font
Builds on: VQGAN FS-Font VQVAE Transformer
Closest papers by meaning
- DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models (AAAI 2024 · similarity 0.89)
- IF-Font: Ideographic Description Sequence-Following Font Generation (NeurIPS 2024 · similarity 0.89)
- VecGlypher: Unified Vector Glyph Generation with Language Models (CVPR 2026 · similarity 0.87)
- Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models (CVPR 2024 · similarity 0.87)
- Decoupling Layout from Glyph in Online Chinese Handwriting Generation (ICLR 2025 · similarity 0.86)
- FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.