Research map / Image generation / Editing and style transfer
Text-Driven Fashion Image Editing with Compositional Concept Learning and Counterfactual Abduction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T-FIT, a text-only fashion image editing framework that combines counterfactual abduction (a LoRA-tuned denoising U-Net) with concept learning via SVD-projected text-prompt directions and weighted multi-concept composition on Stable Diffusion v2.1.
Problem
Text-driven fashion image editing suffers from spurious correlations that cause unintended changes outside the target region, poor non-rigid edits (e.g., sleeve or garment length), and concept omission or confusion when editing multiple concepts at once.
Approach
Infer an exogenous variable for the source image by fine-tuning a LoRA on the denoising U-Net (counterfactual abduction), then learn concept directions as projections onto the span of SVD-derived text-prompt score differences, and compose several weighted concept directions for simultaneous multi-concept edits.
Key results
- Single-concept editing: CLIP-D 0.1433 vs. best baseline 0.0954 (LEDITS++), with CLIP-I 0.9160 tied for best.
- Multi-concept editing: CLIP-D 0.1152 vs. 0.0967 (Turbo-edit); CLIP-I 0.9259, below TexFit (0.9562) and Turbo-edit (0.9355).
- Human preference study with 36 volunteers ranked T-FIT best on both text alignment and image consistency among 6 methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use T-FIT instead of the compared baselines | Fashion image editing · Single concept | CLIP-D: 0.0954 -> 0.1433 | Helped |
| Use T-FIT instead of the compared baselines | Fashion image editing · Multiple concepts | CLIP-D: 0.0967 -> 0.1152 | Helped |
| Use T-FIT instead of the compared baselines | Fashion image editing · Single concept | CLIP-I: 0.9500 -> 0.9160 | Mixed |
| Use T-FIT instead of the compared baselines | Fashion image editing · Multiple concepts | CLIP-I: 0.9562 -> 0.9259 | Mixed |
| Replace counterfactual abduction with null-text inversion (ablation) | T-FIT · Fashion image editing | preservation of source-image details | Hurt |
| Remove the concept learning module (ablation) | T-FIT · Fashion image editing | editability | Hurt |
| Vary annealing parameter κ while using counterfactual abduction alone (ablation) | T-FIT without CLM · Fashion image editing | editability and fidelity balance | No clear effect |
| Adjust individual concept weights (ablation) | T-FIT concept composition · Single- and multi-concept fashion image editing | editing intensity control | Helped |
Surprising findings
- Higher CLIP-I for Turbo-edit and TexFit is attributed to omitting edits or barely changing images, so CLIP-I alone is not a reliable measure of editing quality.
Limitations
- Relies on a self-collected, small evaluation set (30 images in human study; Unsplash images) with limited description of its size.
- Per-image LoRA fine-tuning (100 epochs) is required for each source image.
- Future work noted: incorporating visual information such as style images to learn richer concept representations.
Experimental setup
Datasets: Unsplash fashion images (self-collected)
Benchmarks: Unsplash fashion image editing set (self-collected)
Metrics: CLIP-I CLIP-D human preference ranking
Baselines: LEDITS LEDITS++ TexFit Guide-and-Rescale UltraEdit Turbo-edit
Builds on: Stable Diffusion null-text inversion classifier-free guidance LEDITS++ DDIM
Closest papers by meaning
- TexFit: Text-Driven Fashion Image Editing with Diffusion Models (AAAI 2024 · similarity 0.89)
- FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving (AAAI 2025 · similarity 0.87)
- Pattern Analogies: Learning to Perform Programmatic Image Edits by Analogy (CVPR 2025 · similarity 0.87)
- HieraFashDiff: Hierarchical Fashion Design with Multi-stage Diffusion Models (AAAI 2025 · similarity 0.86)
- Doubly Abductive Counterfactual Inference for Text-based Image Editing (CVPR 2024 · similarity 0.85)
- Concept Lancet: Image Editing with Compositional Representation Transplant (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.