Research map / Image generation / Text-to-image diffusion
RefDiT: Local Attribute Guidance in Reference-Based Image Generation
Rameshwar Mishra, Srikrishna Karanam, Subramanyam Venkata
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SCM attribute-aware triplet conditioning to DreamBooth baseline (M1 to M2) (ablation) | DreamBooth-style LoRA · Ablation study | Attr-SIM: 0.43 -> 0.57 | Improved |
| add SCM conditioning (M1 to M2) (ablation) | DreamBooth-style LoRA · Ablation study | CLIP-I: 0.71 -> 0.77 | Improved |
| add SCM conditioning (M1 to M2) (ablation) | DreamBooth-style LoRA · Ablation study | CLIP-T: 0.29 -> 0.27 | Hurt |
| add triplet-conditioned LoRA training with region consistency loss (M2 to full) (ablation) | RefDiT · Ablation study | Attr-SIM: 0.57 -> 0.70 | Improved |
| add triplet conditioning (M2 to full) (ablation) | RefDiT · Ablation study | CLIP-T: 0.27 -> 0.32 | Improved |
| add triplet conditioning (M2 to full) (ablation) | RefDiT · Identifier-token attention vs SAM masks | mIoU: improves by 0.17 | Improved |
| use alpha 0.5 instead of 0.7 (ablation) | RefDiT · Ablation study | Attr-SIM: 0.68 -> 0.70 | Improved |
| use alpha 0.5 instead of 0.3 (ablation) | RefDiT · Ablation study | Attr-SIM: 0.66 -> 0.70 | Improved |
| use GPT-4o instead of InternVL for SCM extraction (ablation) | RefDiT · Ablation study | Attr-SIM: 0.63 -> 0.70 | Improved |
| use GPT-4o instead of InternVL for SCM extraction (ablation) | RefDiT · Ablation study | CLIP-I: 0.73 -> 0.80 | Improved |
| add SCM conditioning (user study M2 over M1) (ablation) | DreamBooth-style LoRA · User study | preference: 72.8% | Improved |
| add triplet conditioning (user study Ours over M2) (ablation) | RefDiT · User study | preference: 68.7% | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 4 of 4 on most shared results.
- K-LoRA — K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs (CVPR 2025) · better on 3 of 3 shared results
- UnZipLoRA — UnZipLoRA: Separating Content and Style from a Single Image (ICCV 2025) · better on 3 of 3 shared results
- FlowEdit — FlowEdit: Inversion-Free Text-Based Editing Using Pre-Trained Flow Models (ICCV 2025) · better on 2 of 2 shared results
- SDEdit — SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations (ICLR 2022) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Refine-by-Align: Reference-Guided Artifacts Refinement through Semantic Alignment (ICLR 2025 · similarity 0.90)
- RefDrop: Controllable Consistency in Image or Video Generation via Reference Feature Guidance (NeurIPS 2024 · similarity 0.87)
- TFCustom: Customized Image Generation with Time-Aware Frequency Feature Guidance (CVPR 2025 · similarity 0.86)
- Referring Image Editing: Object-level Image Editing via Referring Expressions (CVPR 2024 · similarity 0.86)
- One-Shot Reference-based Structure-Aware Image to Sketch Synthesis (AAAI 2025 · similarity 0.85)
- Zero-shot Image Editing with Reference Imitation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
