AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

Fine-Grained Action Retrieval Through Multiple Parts-of-Speech Embeddings

Michael Wray, Diane Larlus, Gabriela Csurka, Dima Damen

ICCV 2019 · Vision-language and multimodal › Cross-modal learning · approach: Probabilistic embeddings for cross-modal retrieval · 156 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
learn separate verb and noun embeddings and combine with a joint PoS-agnostic embedding (JPoSE) instead of one caption embeddingMMEN · EPIC seen, vtmAP: 14.0 -> 23.2Improved
same, JPoSE vs MMEN (Caption)MMEN · EPIC unseen, tvmAP: 7.7 -> 10.2Improved
add the final joint PoS-agnostic embedding on top of concatenated verb and noun MMENs (ablation)MMEN ([Verb, Noun]) · EPIC seen, vtmAP: 18.7 -> 23.2Improved
embed verb and noun separately and concatenate instead of a single caption embedding (ablation)MMEN · EPIC seen, vtmAP: 14.0 -> 18.7Improved
train PoS-MMENs jointly with PoS-agnostic loss rather than independently (concatenation) (ablation)JPoSE · EPIC seen, vtmAP: 21.5 -> 22.7Improved
use concatenation instead of sum encoding (joint training) (ablation)JPoSE · EPIC seen, vtmAP: 21.0 -> 22.7Improved
learn final embeddings f-hat and g-hat instead of identity (ablation)JPoSE · EPIC seen, vtmAP: 22.7 -> 23.2Improved
train on open vocabulary captions instead of closed vocabulary classes (ablation)JPoSE · EPIC seen, vtmAP: 18.0 -> 23.2Improved
use JPoSE instead of MMEN (Caption) for zero-shot verbsMMEN · EPIC ZSV, vtmAP: 5.77 -> 7.50Improved
use JPoSE instead of MMEN (Caption) for zero-shot nounsMMEN · EPIC ZSN, vtmAP: 4.17 -> 7.68Improved
replace the MLP caption encoder with a GRU (Caption RNN) (ablation)MMEN (Caption) · EPIC seen, vtmAP: 14.0 -> 10.3Mixed
disentangle nouns from caption (JPoSE Caption\Noun, Noun) on MSR-VTTMMEN (Caption) · MSR-VTT text-to-videoR@1: 13.8 -> 14.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy