Research map / Image generation / Text-to-image diffusion
Bridge Diffusion Model: Bridge Chinese Text-to-Image Diffusion Model with English Communities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bridge Diffusion Model (BDM), a frozen English Stable Diffusion backbone plus a trainable Chinese-CLIP-conditioned branch, to build a Chinese text-to-image model compatible with English SD community plugins.
Problem
Text-to-image models trained on English-centric data carry language-related bias and cannot natively handle Chinese semantics, while Chinese models trained from scratch lose compatibility with the English community's checkpoints, LoRA, ControlNet and other plugins.
Approach
A ControlNet-like backbone-branch design: the English SD1.5 backbone stays frozen and supplies the latent space, while a trainable branch with a Chinese CLIP text encoder injects Chinese semantics into that shared latent space; the English prompt is kept empty during training so the branch learns the offset.
Key results
- BDM achieves zero-shot FID 9.93 on MS-COCO 256x256, versus 8.59 for Stable Diffusion 1.5 and 14.70 for ERNIE-ViLG.
- In human evaluation on BDM-870 with seven raters, BDM was preferred over ERNIE-ViLG 2.0 by 54.57% (quality) and 62.65% (alignment), and over Vega AI by 54.62% and 58.79%.
- On a Chinese vs. Caucasian CLIP-score cultural probe, BDM (RealV51) leans more Chinese than RealV51 on the Chinese-culture prompts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BDM with Chinese prompts instead of Stable Diffusion 1.5 on translated COCO captions | BDM · MS-COCO 256×256 zero-shot FID-30k | FID: 8.59 -> 9.93 | Hurt |
| Use BDM with Chinese prompts instead of ERNIE-ViLG 2.0 | BDM · BDM-870 | image quality preference: 54.57%±3.1% | Helped |
| Use BDM with Chinese prompts instead of ERNIE-ViLG 2.0 | BDM · BDM-870 | text-image alignment preference: 62.65%±2.9% | Helped |
| Use BDM with Chinese prompts instead of Vega AI | BDM · BDM-870 | image quality preference: 54.62%±3.1% | Helped |
| Use BDM with Chinese prompts instead of Vega AI | BDM · BDM-870 | text-image alignment preference: 58.79%±3.0% | Helped |
| Add the Chinese semantics branch to RealV51 | RealV51 · Chinese cultural inclination across human, architecture, food, and festival prompts | CLIP score for Chinese concepts: 22.86 -> 23.48; 26.19 -> 28.59; 23.65 -> 24.77; 19.20 -> 23.75 | Helped |
| Increase Chinese branch weight from 0 to 1 (ablation) | BDM with RealisticVisionV51 · computer mouse on table prompt (translated English is ambiguous) | Chinese semantic correctness | Helped |
| Set the English backbone text input to an empty string during training | BDM · BDM training | training convergence | Helped |
Surprising findings
- Leaving the English backbone prompt empty during training is necessary; otherwise training did not converge.
- Using a better English backbone (RealisticVisionV51) improved BDM output quality even though BDM was trained on SD1.5.
Limitations
- Training data is predominantly internal and not released, and the model's Chinese-concept coverage relies on it.
- COCO FID is affected by translation errors and the mismatch between Chinese-data concepts and COCO concepts.
- Human evaluation is limited to seven raters and compares against closed commercial systems.
Experimental setup
Datasets: internal Chinese image-text datasets public English image-text datasets MS-COCO validation set (translated to Chinese)
Benchmarks: MS-COCO 256x256 zero-shot FID-30k BDM-870 human evaluation
Metrics: FID human preference rate CLIP score
Baselines: ERNIE-ViLG 2.0 Vega AI Stable Diffusion 1.5 LDM GLIDE DALL-E 2 eDiff-I RAPHAEL
Builds on: Stable Diffusion ControlNet DDPM Chinese CLIP
Closest papers by meaning
- AltDiffusion: A Multilingual Text-to-Image Diffusion Model (AAAI 2024 · similarity 0.91)
- MuLan: Adapting Multilingual Diffusion Models for Hundreds of Languages with Negligible Cost (ICML 2025 · similarity 0.88)
- Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis (CVPR 2025 · similarity 0.87)
- Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.86)
- Image Regeneration: Evaluating Text-to-Image Model via Generating Identical Image with Multimodal Large Language Models (AAAI 2025 · similarity 0.86)
- Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.