AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Cross-Modal Scene Networks

Yusuf Aytar, Lluís Castrejón, Carl Vondrick, Hamed Pirsiavash, Antonio B. Torralba

TPAMI 2017 · Vision-language and multimodal › Cross-modal learning · approach: Unimodal-to-multimodal representation transfer · 126 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add modality tuning with fixed shared layers (method A)BL-Shared-Upper · CMPlaces retrieval (fc7)mean mAP: 6.5 -> 8.5Improved
add statistical regularization with GMM prior (method B)BL-Shared-Upper · CMPlaces retrieval (fc7)mean mAP: 6.5 -> 6.4No clear effect
combine tuning and StatReg (method C)A: Tune · CMPlaces retrieval (fc7)mean mAP: 8.5 -> 11.9Improved
use GMM instead of single Gaussian for regularizer (ablation)StatReg · CMPlaces retrieval (fc6)mean mAP: 4.2 -> 5.5Improved
add method C alignment, retrieval from fc7 to fc6 layerC: Tune + StatReg (GMM) · CMPlaces retrieval (fc6 vs fc7)mean mAP: BL-Shared-Upper 6.5 -> 3.2 versus C 11.9 -> 11.1Improved
add Gaussian StatReg regularizer to shared-upper networkBL-Shared-Upper · Zero-shot classification, clip artaccuracy: 22.1 -> 50.5Improved
add Gaussian StatReg regularizerBL-Shared-Upper · Zero-shot classification, line drawingsaccuracy: 14.9 -> 24.8Improved
add Gaussian StatReg regularizerBL-Shared-Upper · Zero-shot classification, text descriptionsaccuracy: 43.4 -> 4.2Hurt
add modality tuningBL-Shared-Upper · Zero-shot classification, text descriptionsaccuracy: 43.4 -> 51.0Improved
add StatReg to tuning (method C)A: Modality Tuning · Zero-shot classification, clip artaccuracy: 18.6 -> 16.3Hurt
add tuning and StatReg together (method C)BL-Shared-Upper · Zero-shot retrieval, 55 held-out categoriesmean mAP: 7.3 -> BL; A 7.2, B 6.8Improved
add modality tuning (Free) to shared-upperBL-Shared-Upper · Zero-shot retrieval, 55 held-out categoriesmean mAP: 7.3 -> 7.2No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy