Research map / Vision-language and multimodal / Cross-modal learning
See & Sniff: Learning Visuo-Olfactory Representations
Seongyu Kim, Seungwoo Lee, Hyeonggon Ryu, Joon Son Chung, Arda Senocak
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dense local alignment instead of global CLS-token alignment (ablation) | See & Sniff · SmellNet-V-Source localization | mIoU: 0.5016 -> 0.6099 | Improved |
| add lightweight aligners to local alignment (ablation) | Ours-Local · SmellNet-V-Source localization | mAP: 0.7970 -> 0.8362 | Improved |
| add lightweight aligners to local alignment (ablation) | Ours-Local · SmellNet-V-Test Vision->Smell | Recall@1: 55.40 -> 63.20 | Improved |
| dense local alignment vs global alignment on retrieval (ablation) | See & Sniff vs Ours-Global · SmellNet-V-Test Smell->Vision | Recall@10: 67.50 -> 63.90 | Mixed |
| joint visuo-olfactory model instead of cascaded SmellNet + SAM3 | See & Sniff · SmellNet-V-Source localization | mIoU: 0.3214 -> 0.6456 | Improved |
| use smell-conditioned alignment instead of vision-only DINOv3 attention | See & Sniff · SmellNet-V-Source localization | mIoU: 0.5157 -> 0.6456 | Improved |
| CLIP-style global contrastive training with frozen CLIP-Large | Global-CLIP · SmellNet-V-Source localization | mIoU: 0.2073 vs full square 0.2072 | Hurt |
| add visual supervision (SmellNet-V) to olfactory encoder training | ScentFormer · SmellNet-Test Smell->Smell | Recall@1: 52.54 -> 69.44 | Improved |
| add visual supervision to smell encoder | ScentFormer · SmellNet-Test classification | accuracy | Improved |
| visual supervision effect on per-ingredient classification | See & Sniff · SmellNet-Test classification | accuracy: apple 4.55 -> 77.27; avocado 61.90 -> 28.57 | Mixed |
Closest papers by meaning
- What Images Cannot Say: Language-Guided Olfactory Representation Learning (ECCV 2026 · similarity 0.94)
- MixScentNet: A Multiscale Graph-based Framework for Predicting Scent Mixture Perception (NeurIPS 2026 · similarity 0.88)
- SmellNet: A Large-scale Dataset for Real-world Smell Recognition (ICLR 2026 · similarity 0.86)
- Can Transformers Smell Like Humans? (NeurIPS 2024 · similarity 0.85)
- SCENT: Robust Spatiotemporal Learning for Continuous Scientific Data via Scalable Conditioned Neural Fields (ICML 2025 · similarity 0.82)
- CAV-MAE Sync: Improving Contrastive Audio-Visual Mask Autoencoders via Fine-Grained Alignment (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
