Research map / Image generation / Text-to-image diffusion
Yuan: Yielding Unblemished Aesthetics Through a Unified Network for Visual Imperfections Removal in Generated Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Yuan automatically removes visual imperfections from text-to-image outputs by combining Grounded SAM masking (with logit threshold adjustment), LaMa inpainting, and a fine-tuned GPT caption-guided Stable Diffusion repaint fallback.
Problem
Text-to-image outputs often contain visual flaws such as anatomical errors, misplaced objects, and misplaced text, and existing correction pipelines depend on manual masks.
Approach
Replace manual masking with text-conditioned Grounded SAM mask generation (with a tunable logit threshold), inpaint with LaMa instead of diffusion, and fall back to caption-based repainting with Stable Diffusion via a fine-tuned GPT that rewrites captions when inpainting is insufficient.
Key results
- On ImageNet100, Yuan achieved NIQE 3.0890, BRISQUE 25.6086 and PI 2.0124 (PI best), versus 5.2829 / 32.1852 / 5.9084 for Grounded SAM+SD.
- On Stanford-dogs, Yuan achieved the best NIQE (3.4691), BRISQUE (16.2062) and PI (2.2841) among compared methods.
- On Generated-cats, Yuan reached PI 5.4679 (best) but LaMa had better NIQE (5.0716 vs 5.2465) and BRISQUE was worse for Yuan than LaMa-only variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Yuan instead of +LaMa | ImageNet100 object removal · ImageNet100 | NIQE: 3.0905 -> 3.0890 | Helped |
| Use Yuan instead of +LaMa | ImageNet100 object removal · ImageNet100 | BRISQUE: 24.7853 -> 25.6086 | Hurt |
| Use Yuan instead of +LaMa | ImageNet100 object removal · ImageNet100 | PI: 2.0921 -> 2.0124 | Helped |
| Use Yuan instead of +LaMa | Stanford-dogs object removal · Stanford-dogs | NIQE: 4.0785 -> 3.4691 | Helped |
| Use Yuan instead of +LaMa | Stanford-dogs object removal · Stanford-dogs | BRISQUE: 22.0275 -> 16.2062 | Helped |
| Use Yuan instead of +LaMa | Stanford-dogs object removal · Stanford-dogs | PI: 2.6190 -> 2.2841 | Helped |
| Use Yuan instead of +LaMa | Generated-cats object removal · Generated-cats | NIQE: 5.0716 -> 5.2465 | Hurt |
| Use Yuan instead of +LaMa | Generated-cats object removal · Generated-cats | BRISQUE: 45.6096 -> 39.4333 | Helped |
| Use Yuan instead of +LaMa | Generated-cats object removal · Generated-cats | PI: 5.5097 -> 5.4679 | Helped |
| Adjust buffer threshold b by dataset instead of using one fixed setting (ablation) | Yuan · ImageNet100, Stanford-dogs, and Generated-cats | NIQE | Helped |
| Adjust logit threshold t for mask generation (ablation) | Yuan · ImageNet100, Stanford-dogs, and Generated-cats | NIQE | Mixed |
Surprising findings
- Yuan's BRISQUE on Generated-cats (39.43) is worse than Grounded SAM+SD's best-in-column value (37.44) and LaMa-only (45.61) is worse still, showing mixed results across metrics.
- Authors report that LaMa inpainting outperforms diffusion-based inpainting for preserving original image characteristics, contrary to the common assumption that diffusion gives better fills.
Limitations
- Struggles with rendering human hands accurately in generated images.
- Refinement can introduce unintended content, requiring additional rounds that cost resources.
- Evaluation uses only no-reference quality metrics (NIQE, BRISQUE, PI) without task-level or human assessment of imperfection removal.
Experimental setup
Datasets: ImageNet100 Stanford Dogs Generated-cats
Benchmarks: ImageNet100 Stanford Dogs Generated-cats
Metrics: NIQE BRISQUE PI inference time
Baselines: Grounded SAM + Stable Diffusion Grounded SAM + LaMa
Builds on: Grounded SAM LaMa Florence-2 Stable Diffusion
Closest papers by meaning
- ArtiFade: Learning to Generate High-quality Subject from Blemished Images (CVPR 2025 · similarity 0.85)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.85)
- Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator (CVPR 2025 · similarity 0.85)
- MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models (NeurIPS 2025 · similarity 0.85)
- Alchemist: Turning Public Text-to-Image Data into Generative Gold (NeurIPS 2025 · similarity 0.85)
- Focus-N-Fix: Region-Aware Fine-Tuning for Text-to-Image Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.