Research map / Vision-language and multimodal / Human and embodied modeling
Learning Program Representations for Food Images and Cooking Recipes
Dim P. Papadopoulos, Enrique Mora, Nadiia Chepurko, Kuan Wei Huang, Ferda Ofli, Antonio Torralba
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add program prediction losses to joint image-recipe training (ablation) | Image-recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 53.5 -> 58.6 | Improved |
| Increase encoder depth from 8 to 12 layers (ablation) | Full recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 58.6 -> 66.9 | Improved |
| Use average-pooled ViT features instead of the CLS token (ablation) | Image-recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 56.1 -> 58.6 | Improved |
| Increase program-loss weights from 0.1 to 1 (ablation) | Image-recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 58.6 -> 58.0 | Hurt |
| Increase program-loss weights from 0.1 to 10 (ablation) | Image-recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 58.6 -> 56.4 | Hurt |
| Add ingredients to the title representation (ablation) | Recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 24.0 -> 43.6 | Improved |
| Use the full recipe rather than title and ingredients (ablation) | Recipe retrieval model · Recipe1M test, image-to-recipe | R@1: 43.6 -> 53.5 | Improved |
| Generate cooking programs instead of raw instruction text | Food image-conditioned prediction · Food image program prediction | Ingredients F1: 28.5 -> 53.5 | Improved |
| Generate cooking programs instead of raw instruction text | Recipe-conditioned prediction · Cooking recipe program prediction | Ingredients F1: 41.6 -> 75.5 | Improved |
| Use minimum cross-entropy over valid program sequences instead of standard cross-entropy (ablation) | Image-conditioned program decoder · Food image program prediction | Full graph GED: 72.1 -> 67.2 | Improved |
| Use minimum cross-entropy over valid program sequences instead of standard cross-entropy (ablation) | Recipe-conditioned program decoder · Cooking recipe program prediction | Full graph GED: 19.1 -> 16.8 | Improved |
Closest papers by meaning
- Learning Cross-Modal Embeddings for Cooking Recipes and Food Images (CVPR 2017 · similarity 0.91)
- Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images (TPAMI 2019 · similarity 0.90)
- CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrieval (AAAI 2021 · similarity 0.90)
- Structure-Aware Generation Network for Recipe Generation from Images (ECCV 2020 · similarity 0.89)
- Pic2Prep: A Multimodal Conversational Agent for Cooking Assistance (AAAI 2025 · similarity 0.88)
- OVFoodSeg: Elevating Open-Vocabulary Food Image Segmentation via Image-Informed Textual Representation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
