Research map / Image generation / Editing and style transfer
OmniConsistency: Learning Style-Agnostic Consistency from Paired Stylization Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OmniConsistency, a DiT (FLUX) consistency plugin trained on 2,600 GPT-4o-generated aligned stylization pairs with a two-stage decoupled training and rolling LoRA bank, for style-agnostic image-to-image stylization.
Problem
Image-to-image stylization with style LoRAs suffers from style degradation and poor content/identity/detail consistency in complex scenes, with a gap between open-source methods and GPT-4o.
Approach
Train a lightweight condition-branch-only consistency LoRA on paired stylization data in-context (original image as implicit condition, causal attention), decoupled from style LoRAs that are rolled in from a pre-trained LoRA bank during training so the consistency module does not absorb style.
Key results
- Ours achieves FID 39.2 and CMMD 0.145 on style consistency, best among compared baselines (Flux I2I 44.4/0.168), and GPT-4o score 4.64 for style.
- Content consistency GPT-4o score 4.52 vs GPT-4o 4.57; CLIP-S 0.321 highest among methods.
- Overhead of only 4.6% GPU memory and 5.3% inference time over Flux T2I at 1024x1024 with 24 steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the rolling LoRA Bank training with multiple style LoRAs (ablation) | OmniConsistency · 100-image stylization benchmark | Style consistency GPT-4o score: 4.14 -> 4.64 | Helped |
| Add decoupled style and consistency training (ablation) | OmniConsistency · 100-image stylization benchmark | Content DreamSim: 0.363 -> 0.278 | Helped |
| Add rolling LoRA Bank training with multiple style LoRAs (ablation) | OmniConsistency · 100-image stylization benchmark | Content CLIP-I: 0.762 -> 0.741 | Hurt |
| Use OmniConsistency instead of Flux T2I + Redux | Flux 1.0 dev stylization · 100-image stylization benchmark | Style consistency GPT-4o score: 4.49 -> 4.64 | Helped |
| Use OmniConsistency instead of Flux I2I + Redux | Flux 1.0 dev stylization · 100-image stylization benchmark | Content DreamSim: 0.284 -> 0.278 | Helped |
Surprising findings
- Flux I2I + Redux achieves the highest CLIP-I content score, attributed to limited stylization strength rather than better consistency.
Limitations
- Struggles to preserve non-English text due to FLUX backbone
- Artifacts in small facial and hand regions
- Inconsistent number of people in group photos and complex scenes
Experimental setup
Datasets: GPT-4o-generated stylized image pairs (22 styles, 2,600 pairs) LibLibAI style LoRAs for evaluation
Benchmarks: 100-image custom image-to-image stylization benchmark (5 unseen LibLibAI style LoRAs) FID/CMMD over 10 styles
Metrics: FID CMMD DreamSim CLIP-I GPT-4o score CLIP-S GPU memory inference time user preference
Baselines: Flux I2I (SDEdit) Flux I2I + Redux Flux T2I + Redux Flux I2I + ControlNet Flux T2I + ControlNet GPT-4o
Builds on: FLUX LoRA ControlNet OminiControl
Closest papers by meaning
- OmniStyle: Filtering High Quality Style Transfer Data at Scale (CVPR 2025 · similarity 0.89)
- Conditional Balance: Improving Multi-Conditioning Trade-Offs in Image Generation (CVPR 2025 · similarity 0.87)
- Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles (NeurIPS 2025 · similarity 0.86)
- Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion models (NeurIPS 2024 · similarity 0.86)
- One-Shot Structure-Aware Stylized Image Synthesis (CVPR 2024 · similarity 0.85)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.