Research map / Vision-language and multimodal / Cross-modal learning
Language-Agnostic Visual-Semantic Embeddings
Jonatas Wehrmann, Douglas M. Souza, Mauricio A. Lopes, Rodrigo C. Barros
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the non-local module (ablation) | LIWE · Flickr30k image-to-text | R@1: 61.8 -> 66.4 | Improved |
| use non-linear NLM instead of linear NLM (ablation) | LIWE · Flickr30k image-to-text | R@1: 60.7 -> 66.4 | Improved |
| reduce input dimensionality before NLM query (ablation) | LIWE · Flickr30k image-to-text | R@1: 56.3 -> 66.4 | Improved |
| use proposed loss instead of loss from [19] (ablation) | CLMR · Flickr30k image-to-text | R@1: 60.4 -> 65.8 | Improved |
| use proposed loss instead of loss from [19] (ablation) | LIWE(256,256) · Flickr30k image-to-text | R@1: 63.2 -> 65.3 | Improved |
| use proposed loss instead of loss from [10] (ablation) | CLMR and LIWE · Flickr30k | R@1: Diverges -> 65.8 | Improved |
| concatenate pretrained Glove to character-based embeddings | LIWE · MS COCO test | recall sum (Σ): 498.6 -> 507.7 | Improved |
| concatenate pretrained Glove to character-based embeddings | LIWE · Flickr30k test | recall sum (Σ): 458.1 -> 474.3 | Improved |
| replace word embeddings with character-based LIWE | CLMR · Multi30k German test | recall sum (Σ): 357.3 -> 392.0 | Improved |
| replace word embeddings with character-based LIWE | CLMR · YJ Captions test | recall sum (Σ): 451.4 -> 448.4 | Hurt |
| replace word embeddings with character-based LIWE | CLMR · MS COCO test | recall sum (Σ): 500.3 -> 498.6 | No clear effect |
| use LIWE instead of BERT-Multilingual | CLMR-style co-training · Multi30k English image-to-text | R@1: 62.0 -> 64.4 | Improved |
Closest papers by meaning
- CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer (AAAI 2024 · similarity 0.90)
- Adaptive Cross-Modal Prototypes for Cross-Domain Visual-Language Retrieval (CVPR 2021 · similarity 0.90)
- Image-Text Embedding Learning via Visual and Textual Semantic Reasoning (TPAMI 2022 · similarity 0.90)
- Dynamic Adapter with Semantics Disentangling for Cross-lingual Cross-modal Retrieval (AAAI 2025 · similarity 0.89)
- Adaptive Prompt-Based Semantic Embedding with Inspire Potential of Implicit Knowledge for Cross-Modal Retrieval (AAAI 2025 · similarity 0.89)
- WikiDO: A New Benchmark Evaluating Cross-Modal Retrieval for Vision-Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
