Research map / Vision-language and multimodal / Cross-modal learning
Product1M: Towards Weakly Supervised Instance-Level Product Retrieval via Cross-Modal Pretraining
Xunlin Zhan, Yangxin Wu, Xiao Dong, Yunchao Wei, Minlong Lu, Yichi Zhang, Hang Xu, Xiaodan Liang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add image-text matching (ITM) pretext task (ablation) | CAPTURE masked pretraining · Product1M pretext tasks (N=100) | mAP: 72.1 -> 70.2 | Hurt |
| add cross-modal contrastive loss (ablation) | CAPTURE masked pretraining · Product1M pretext tasks (N=100) | mAP: 72.1 -> 73.3 | Improved |
| concatenate Text/Visual Transformer features with Co-Transformer features for retrieval (ablation) | CAPTURE with contrastive loss · Product1M pretext tasks (N=100) | mAP: 73.3 -> 74.6 | Improved |
| concatenate features without contrastive loss (ablation) | CAPTURE masked pretraining · Product1M pretext tasks (N=100) | mAP: 72.1 -> 71.9 | Hurt |
| add Cross-Transformer layers (ablation) | CAPTURE · Product1M layer configuration (N=100) | mAP: 73.8 -> 74.6 | Improved |
| add Co-Transformer layers (ablation) | CAPTURE · Product1M layer configuration (N=100) | mAP: 73.2 -> 74.6 | Improved |
| add Text/Visual Transformer layers (ablation) | CAPTURE · Product1M layer configuration (N=100) | mAP: 69.3 -> 74.6 | Improved |
| double depth from 12 to 24 layers (CAPTURE-L) (ablation) | CAPTURE · Product1M layer configuration (N=100) | mAP: 74.6 -> 74.7 | No clear effect |
| increase depth from 6 to 12 layers (ablation) | CAPTURE · Product1M layer configuration (N=100) | mAP: 67.7 -> 74.6 | Improved |
| use augmentation-trained RPN instead of Faster R-CNN pretrained on Visual Genome (ablation) | CAPTURE · Product1M detection impact | mAP@10: 70.36 -> 74.63 | Improved |
| use detected instances instead of one whole-image box (ablation) | CAPTURE · Product1M detection impact | mAP@10: 60.03 -> 74.63 | Improved |
| use ground truth boxes instead of RPN boxes (ablation) | CAPTURE · Product1M annotated subset | mAP@10: 73.36 -> 77.79 | Improved |
Closest papers by meaning
- Entity-Graph Enhanced Cross-Modal Pretraining for Instance-level Product Retrieval (TPAMI 2023 · similarity 0.92)
- Learning Instance-Level Representation for Large-Scale Multi-Modal Pretraining in E-Commerce (CVPR 2023 · similarity 0.88)
- Cross-Domain Product Representation Learning for Rich-Content E-Commerce (ICCV 2023 · similarity 0.87)
- Transformer-Empowered Multi-Modal Item Embedding for Enhanced Image Search in E-commerce (AAAI 2024 · similarity 0.86)
- Beyond Global Similarity: Multi-Conditional Retrieval for Fine-Grained Cross-Modal Understanding (CVPR 2026 · similarity 0.85)
- Easy Regional Contrastive Learning of Expressive Fashion Representations (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
