Research map / Vision-language and multimodal / CLIP and zero-shot
Mining on Heterogeneous Manifolds for Zero-Shot Cross-Modal Image Retrieval
Fan Yang, Zheng Wang, Jing Xiao, Shin'ichi Satoh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| build graph on single-modal (heterogeneous) manifolds instead of cross-modal (homogeneous) space (ablation) | Bi-directional random walk · RegDB | mAP: 29.3% -> 32.1% | Improved |
| build graph on single-modal (heterogeneous) manifolds instead of cross-modal (homogeneous) space (ablation) | Bi-directional random walk · SYSU-MM01 | mAP: 34.9% -> 38.0% | Improved |
| build graph on single-modal (heterogeneous) manifolds instead of cross-modal space (ablation) | Bi-directional random walk · MNIST->SVHN digits 7-9 | mAP: 51.7% -> 60.3% | Improved |
| build graph on single-modal (heterogeneous) manifolds instead of cross-modal space (ablation) | Bi-directional random walk · SVHN->MNIST digits 7-9 | mAP: 52.0% -> 58.4% | Improved |
| build graph on single-modal (heterogeneous) manifolds instead of cross-modal space (ablation) | Bi-directional random walk · RegDB | Rank-10: 53.1% -> 47.0% | Hurt |
| build graph on single-modal (heterogeneous) manifolds instead of cross-modal space (ablation) | Bi-directional random walk · MNIST->SVHN digits 7-9 | Rank-10: 96.4% -> 92.2% | Hurt |
| add heterogeneous bi-directional random walk on top of k-NN baseline | ResNet50 two-stream model · RegDB | mAP: 23.9% -> 32.1% | Improved |
| add heterogeneous bi-directional random walk on top of k-NN baseline | ResNet50 two-stream model · SYSU-MM01 | mAP: 26.7% -> 38.0% | Improved |
| add heterogeneous bi-directional random walk on top of k-NN baseline | ResNet50 two-stream model · RegDB | Rank-10: 44.4% -> 47.0% | Mixed |
| normalize similarity scores per query before ensembling both directions (ablation) | Bi-directional random walk · SVHN->MNIST digits 7-9 | mAP | No clear effect |
| combine both search directions (0<lambda<1) instead of one direction only (ablation) | Bi-directional random walk · SVHN->MNIST digits 7-9 | mAP | Improved |
Closest papers by meaning
- Cross-Modal Center Loss for 3D Cross-Modal Retrieval (CVPR 2021 · similarity 0.87)
- Probabilistic Embeddings for Cross-Modal Retrieval (CVPR 2021 · similarity 0.87)
- Online Asymmetric Similarity Learning for Cross-Modal Retrieval (CVPR 2017 · similarity 0.87)
- WikiDO: A New Benchmark Evaluating Cross-Modal Retrieval for Vision-Language Models (NeurIPS 2024 · similarity 0.85)
- Asymmetric Mutual Alignment for Unsupervised Zero-Shot Sketch-Based Image Retrieval (AAAI 2024 · similarity 0.85)
- Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
