Research map / Image generation / Editing and style transfer
A Multi-Style Chinese Characters Writing Intelligent Tool Based on Small-scale Training Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a real-time interactive tool that turns user handwriting into stylized Chinese character art using a Stable Diffusion img2img workflow with 10 LoRA style models trained on about 50 scanned images per style.
Problem
Mastering diverse Chinese character art styles requires extensive practice, limiting public participation; text-to-image tools do not support the act of writing and miss traditional stylistic essence.
Approach
Combine user handwritten stroke input with a diffusion img2img pipeline and multiple style LoRAs trained on small-scale (about 50 images per style) scanned datasets with manually corrected tags, plus per-style default parameters, to produce personalized stylized characters in real time.
Key results
- Exhibited at three venues attracting 120,000 visitors.
- Survey of 137 visitors: most rated their creations 4-5 out of 5.
- Qualitatively, img2img with ControlNet and Midjourney/DALL·E fail to capture the Chinese character art style, while the LoRA-based tool preserves it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use handwritten stroke input with diffusion and style-specific LoRA models instead of a Stable Diffusion img2img + ControlNet workflow without LoRA | Chinese character writing tool · Chinese character art generation | traditional Chinese character art style fidelity | Helped |
| Use handwritten Chinese character strokes with a diffusion model and 10 LoRA models trained on small-scale data instead of large models | Chinese character writing tool · Chinese character art generation | stylistic effects | Helped |
| Generate from image-based input with Midjourney or DALL·E | Chinese character art generation · Chinese character art generation | Chinese character form fidelity | Hurt |
Limitations
- Style diversity limited by the time needed for dataset creation and training.
- Evaluation is qualitative and based on a small informal visitor survey (137 people), with no quantitative metrics.
- Manual tag adjustment is required because off-the-shelf taggers produce inaccurate tags for this domain.
Experimental setup
Datasets: Chinese character art images scanned from old books and collections (50 per category)
Metrics: qualitative visual comparison user survey ratings (1-5 scale)
Baselines: Stable Diffusion img2img with ControlNet (no LoRA) Midjourney DALL·E D·design
Builds on: LoRA Stable Diffusion ControlNet ComfyUI
Closest papers by meaning
- Decoupling Layout from Glyph in Online Chinese Handwriting Generation (ICLR 2025 · similarity 0.85)
- Stylebreeder: Exploring and Democratizing Artistic Styles through Text-to-Image Models (NeurIPS 2024 · similarity 0.82)
- ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank (AAAI 2024 · similarity 0.82)
- Towards Automated Chinese Ancient Character Restoration: A Diffusion-Based Method with a New Dataset (AAAI 2024 · similarity 0.82)
- GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing (CVPR 2025 · similarity 0.82)
- DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.