Research map / Image generation / Autoregressive and latent generation
DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeepCalliFont, a dual-modality (glyph image + writing trajectory) few-shot Chinese calligraphy font synthesis network with feature recombination, a differentiable rasterization loss, and two-phase pre-training.
Problem
Few-shot Chinese font generation fails on calligraphy fonts with connected strokes, because their glyphs violate the component and stroke-order consistency assumptions used by prior-knowledge-guided methods.
Approach
Jointly model glyph images and writing trajectories in two branches, integrating them via an attention-based Image Feature Recombination module, a distillation-restoration step that maps both style features into a shared space, and a differentiable rasterization loss that keeps trajectories inside glyphs; pre-train each branch on uni-modality data first.
Key results
- On calligraphy fonts, DeepCalliFont achieves FID 14.6 and LPIPS 0.1718 versus 167.2 and 0.2714 for the best baseline WriteLikeYou.
- On regular fonts, it reaches MAE 0.1242, LPIPS 0.1837 and DTW 1.347, best on these metrics, while FontRL has lower FID (40.7 vs 105.9).
- Ablation shows removing the uni-modality pre-training raises FID from 14.6 to 43.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeepCalliFont instead of AGIS-Net | Calligraphy font synthesis · Calligraphy fonts | MAE: 0.1383 -> 0.1254 | Helped |
| Use DeepCalliFont instead of WriteLikeYou | Calligraphy font synthesis · Calligraphy fonts | FID: 167.2 -> 14.6 | Helped |
| Use DeepCalliFont instead of FontRL | Regular font synthesis · Regular fonts | MAE: 0.1350 -> 0.1242 | Helped |
| Use DeepCalliFont instead of FontRL | Regular font synthesis · Regular fonts | FID: 40.7 -> 105.9 | Hurt |
| Use DeepCalliFont instead of FontRL | Regular font synthesis · Regular fonts | LPIPS: 0.2588 -> 0.1837 | Helped |
| Use DeepCalliFont instead of WriteLikeYou | Regular font synthesis · Regular fonts | DTW: 5.920 -> 1.347 | Helped |
| Add IFR (ablation) | DeepCalliFont · Calligraphy fonts | FID: 23.7 -> 14.6 | Helped |
| Add distillation-restoration (ablation) | DeepCalliFont · Calligraphy fonts | FID: 24.6 -> 14.6 | Helped |
| Add differentiable rasterization loss (ablation) | DeepCalliFont · Calligraphy fonts | FID: 24.6 -> 14.6 | Helped |
| Add InfoNCE loss (ablation) | DeepCalliFont · Calligraphy fonts | FID: 27.0 -> 14.6 | Helped |
| Add deep metric learning (ablation) | DeepCalliFont · Calligraphy fonts | FID: 25.4 -> 14.6 | Helped |
| Add the sequence branch and dual-modality components (ablation) | DeepCalliFont · Calligraphy fonts | FID: 21.4 -> 14.6 | Helped |
Surprising findings
- FontRL achieves a lower FID (40.7) than DeepCalliFont (105.9) on regular fonts, which the paper does not highlight as a win.
Limitations
- Few-shot font generation is ill-posed, so visually pleasing outputs may differ from ground truth.
- Connected-stroke labels are pseudo-annotated via a distance threshold rather than human labels.
- Method requires writing trajectory data, limiting applicability to fonts without trajectories.
Experimental setup
Datasets: CASIA Online Chinese Handwriting Databases 42 fonts (3,000 glyph-trajectory pairs each) 251 fonts Liu and Lian regular font dataset (30 fonts) 10 calligraphy fonts collected by authors
Benchmarks: calligraphy font test set (10 fonts) regular font test set (30 fonts)
Metrics: MAE FID LPIPS DTW
Baselines: zi2zi EMD AGIS-Net MX-Font DG-Font FontRL WriteLikeYou
Builds on: zi2zi MX-Font WriteLikeYou InfoNCE deep metric learning
Closest papers by meaning
- VQ-FONT: Few-Shot Font Generation with Structure-Aware Enhancement and Quantization (AAAI 2024 · similarity 0.89)
- Decoupling Layout from Glyph in Online Chinese Handwriting Generation (ICLR 2025 · similarity 0.87)
- VecGlypher: Unified Vector Glyph Generation with Language Models (CVPR 2026 · similarity 0.87)
- IF-Font: Ideographic Description Sequence-Following Font Generation (NeurIPS 2024 · similarity 0.87)
- GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing (CVPR 2025 · similarity 0.86)
- Generate Like Experts: Multi-Stage Font Generation by Incorporating Font Transfer Process into Diffusion Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.