Research map / Vision-language and multimodal / Human and embodied modeling
Learning Joint Visual Semantic Matching Embeddings for Language-guided Retrieval
Yanbei Chen, Loris Bazzani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add JVSM privileged-information losses to TIRG | TIRG · Fashion-200k | R@1: 15.1 -> 19.0 | Improved |
| JVSM instead of TIRG* | TIRG* · UT-Zap50K | R@10: 25.4 -> 37.1 | Improved |
| JVSM instead of TIRG* | TIRG* · Fashion-IQ Dress | R@10: 7.3 -> 10.7 | Improved |
| add visual semantic embedding loss Lvse and Lim to language-guided retrieval baseline (Lim only) (ablation) | TIRG* · Fashion-200k language-guided image retrieval | R@10: 41.9 -> 44.0 | Improved |
| add compositional text matching loss Ltm (ablation) | JVSM (Lvse + Lim) · Fashion-200k language-guided image retrieval | R@1: 15.6 -> 19.0 | Improved |
| add compositional text matching loss Ltm (ablation) | JVSM (Lvse + Lim) · Fashion-200k language-guided text retrieval | R@1: 34.2 -> 50.4 | Improved |
| add compositional losses to VSE-only model (ablation) | VSE (Lvse) · Fashion-200k text-to-image retrieval | R@1: 22.7 -> 23.4 | Improved |
| add Lim only to VSE model for text-to-image retrieval (ablation) | VSE (Lvse) · Fashion-200k text-to-image retrieval | R@1: 22.7 -> 21.3 | Hurt |
| use semi-hard negative mining instead of hardest negative mining (ablation) | VSE training · Fashion-200k | convergence | Improved |
| add two-stage training (VSE first, then full loss) (ablation) | JVSM · Fashion-200k | loss minimum | Improved |
| use soft semantic matching instead of semi-hard matching when privileged info is partial (ablation) | JVSM · Fashion-IQ | performance | Improved |
| use soft semantic matching instead of semi-hard matching when fully annotated (ablation) | JVSM · fully annotated datasets | performance | No clear effect |
Closest papers by meaning
- Dual Compositional Learning in Interactive Image Retrieval (AAAI 2021 · similarity 0.89)
- Image-Text Embedding Learning via Visual and Textual Semantic Reasoning (TPAMI 2022 · similarity 0.89)
- FashionVLP: Vision Language Transformer for Fashion Retrieval With Feedback (CVPR 2022 · similarity 0.89)
- FashionERN: Enhance-and-Refine Network for Composed Fashion Image Retrieval (AAAI 2024 · similarity 0.88)
- Easy Regional Contrastive Learning of Expressive Fashion Representations (NeurIPS 2024 · similarity 0.88)
- Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
