Research map / Vision-language and multimodal / Cross-modal learning
Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images
Javier Marín, Aritro Biswas, Ferda Ofli, Nicholas Hynes, Amaia Salvador, Yusuf Aytar, Ingmar G. Weber, Antonio B. Torralba
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add semantic (classification) regularization to the joint embedding (ablation) | joint embedding, ResNet-50 · Recipe1M im2recipe (1K) | medR: 7.2 -> 5.2 | Improved |
| add semantic regularization (ablation) | joint embedding, ResNet-50 · Recipe1M recipe2im (1K) | medR: 6.9 -> 5.1 | Improved |
| add semantic regularization on top of fine-tuning (ablation) | joint embedding, VGG-16 · Recipe1M im2recipe (1K) | medR: 12.1 -> 8.2 | Improved |
| fine-tune the vision network instead of freezing it (ablation) | joint embedding, ResNet-50 · Recipe1M im2recipe (1K) | medR: 7.9 -> 7.2 | Improved |
| fine-tune the vision network instead of freezing it (ablation) | joint embedding, VGG-16 · Recipe1M im2recipe (1K) | medR: 15.3 -> 12.1 | Improved |
| use ResNet-50 instead of VGG-16 visual features (ablation) | joint embedding with fine-tuning and semantic reg. · Recipe1M im2recipe (1K) | medR: 8.2 -> 5.2 | Improved |
| use ingredient word2vec and skip-instructions in CCA instead of generic word2vec and skip-thoughts (ablation) | CCA baseline · Recipe1M im2recipe (1K) | medR: 25.2 -> 15.7 | Improved |
| train on Recipe1M+ instead of Recipe1M | joint embedding · Recipe1M+ test set, im2recipe | medR: 13.6 -> 8.6 | Improved |
| train on Recipe1M+ instead of Recipe1M | joint embedding · Recipe1M test set, im2recipe | medR: 5.1 -> 5.7 | No clear effect |
| train on Recipe1M+ instead of Recipe1M | joint embedding · Recipe1M+ test set, recipe2im | medR: 11.9 -> 6.8 | Improved |
| train on Recipe1M+ instead of Recipe1M | joint embedding · Food-101 im2recipe | medR: 17.35 -> 10.15 | Improved |
| train on Recipe1M+ instead of Recipe1M | joint embedding · Food-101 recipe2im | medR: 4.75 -> 2.60 | Improved |
Closest papers by meaning
- Learning Cross-Modal Embeddings for Cooking Recipes and Food Images (CVPR 2017 · similarity 0.98)
- CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval (AAAI 2021 · similarity 0.93)
- Revamping Cross-Modal Recipe Retrieval With Hierarchical Transformers and Self-Supervised Learning (CVPR 2021 · similarity 0.92)
- Enhancing Recipe Retrieval with Foundation Models: A Data Augmentation Perspective (ECCV 2024 · similarity 0.92)
- Pic2Prep: A Multimodal Conversational Agent for Cooking Assistance (AAAI 2025 · similarity 0.88)
- OVFoodSeg: Elevating Open-Vocabulary Food Image Segmentation via Image-Informed Textual Representation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
