Research map / Vision-language and multimodal / Human and embodied modeling
FashionERN: Enhance-and-Refine Network for Composed Fashion Image Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FashionERN, which enhances the text (modifier) encoder with reference-injection and target-alignment branches and refines visual tokens with modifier-guided and self-guided attention, to counter reference dominance in composed fashion image retrieval.
Problem
Composed fashion image retrieval with pre-trained symmetric encoders (e.g., CLIP) suffers from reference dominance: retrieval results are dominated by the reference image while the brief modifier text is overlooked.
Approach
Triple-branch Modifier Enhancement injects reference-image semantics into the modifier embedding and aligns it with target-image embeddings via KL divergence; Dual-guided Vision Refinement filters reference patch features first by the modifier and then by self-attention, and fuses them with the enhanced text and reference embeddings.
Key results
- On FashionIQ under the VAL protocol with ResNet, FashionERN reaches 62.49 mean recall versus 50.68 for Comquery, an 11.81% relative gain.
- On FashionIQ with the original protocol and ViT-B/16, FashionERN reaches 62.62 mean recall versus 58.84 (FAME-MiL 55.36), about 7.26% higher than FAME-MiL.
- On CIRR, FashionERN reaches 74.85 mean recall, 5.26% above the best prior method, and on Shoes 68.65 mean recall, the best reported.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the text encoder with one trained on LAION (ablation) | FashionIQ baseline with ResNet · FashionIQ | Mean: 50.03 -> 50.53 | Helped |
| add TME (ablation) | FashionIQ baseline with ResNet · FashionIQ | Mean: 50.03 -> 53.22 | Helped |
| add DVR to the TME model (ablation) | FashionIQ model with ResNet · FashionIQ | Mean: 53.22 -> 55.63 | Helped |
| replace the text encoder with one trained on LAION (ablation) | FashionIQ baseline with ViT-B/16 · FashionIQ | Mean: 55.77 -> 56.62 | Helped |
| add TME (ablation) | FashionIQ baseline with ViT-B/16 · FashionIQ | Mean: 55.77 -> 60.16 | Helped |
| add DVR to the TME model (ablation) | FashionIQ model with ViT-B/16 · FashionIQ | Mean: 60.16 -> 62.62 | Helped |
Surprising findings
- A stronger text encoder (LAION-trained) alone gives only small gains, while the TME model gives larger improvements, indicating the gain is not just from a better text encoder.
Limitations
- Authors note the need to reduce time complexity for practical retrieval.
- Method is limited to single-round retrieval; multi-round interaction is not addressed.
Experimental setup
Datasets: FashionIQ Fashion200K CIRR Shoes
Benchmarks: FashionIQ Fashion200K CIRR Shoes
Metrics: Recall@10 Recall@50 Recall@5 Recall@1 Mean recall Dominance Ratio
Baselines: CLIP4Cir MGUR Comquery FashionVLP FashionSAP Css-Net FAME-MiL FashionViL DCNet ARTEMIS
Builds on: CLIP4Cir CLIP KaleidoBERT FashionVLP
Closest papers by meaning
- Easy Regional Contrastive Learning of Expressive Fashion Representations (NeurIPS 2024 · similarity 0.90)
- Decomposing Semantic Shifts for Composed Image Retrieval (AAAI 2024 · similarity 0.86)
- TexFit: Text-Driven Fashion Image Editing with Diffusion Models (AAAI 2024 · similarity 0.85)
- FashionTailor: Controllable Clothing Editing for Human Images with Appearance Preserving (AAAI 2025 · similarity 0.85)
- Learning to Align Generative Appearance Priors for Fine-grained Image Retrieval (NeurIPS 2026 · similarity 0.85)
- Instance-Level Composed Image Retrieval (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.