Research map / Vision-language and multimodal / Human and embodied modeling
Hierarchical Multimodal LSTM for Dense Visual-Semantic Embedding
Zhenxing Niu, Mo Zhou, Le Wang, Xinbo Gao, Gang Hua
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HM-LSTM instead of NIC | image-sentence ranking · Flickr8K image annotation | R@1: 20 -> 27.7 | Improved |
| use HM-LSTM instead of DeepVS | image-sentence ranking · Flickr8K image annotation | R@1: 16.5 -> 27.7 | Improved |
| use HM-LSTM instead of DeepSP (mean-vector) | image-sentence ranking · Flickr30K image annotation | R@1: 35.7 -> 38.1 | Improved |
| use HM-LSTM instead of DeepSP | image-sentence ranking · Flickr30K image search | R@1: 25.1 -> 27.7 | Improved |
| use HM-LSTM instead of m-RNN-vgg | image-sentence ranking · Flickr30K image annotation | R@1: 35.4 -> 38.1 | Improved |
| use HM-LSTM instead of DeepSP | image-sentence ranking · MS-COCO (1000 test images) image annotation | R@1: 40.7 -> 43.9 | Improved |
| use HM-LSTM instead of DeepSP | image-sentence ranking · MS-COCO (1000 test images) image search | R@1: 33.5 -> 36.1 | Improved |
| use HM-LSTM instead of m-RNN-vgg | region-phrase ranking · MS-COCO-region | R@1: 8.1 -> 10.8 | Improved |
| use HM-LSTM instead of DeepVS | region-phrase ranking · MS-COCO-region | median rank: 64 -> 42 | Improved |
| use HM-LSTM instead of m-RNN-vgg on median rank of image search | image-sentence ranking · MS-COCO (1000 test images) image search | median rank: 3 -> 3 | No clear effect |
Closest papers by meaning
- Jointly Modeling Embedding and Translation to Bridge Video and Language (CVPR 2016 · similarity 0.90)
- Training and Evaluating Multimodal Word Embeddings with Large-scale Web Annotated Images (NeurIPS 2016 · similarity 0.89)
- Recurrent Multimodal Interaction for Referring Image Segmentation (ICCV 2017 · similarity 0.89)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.89)
- Deep Visual-Semantic Alignments for Generating Image Descriptions (TPAMI 2016 · similarity 0.88)
- UVU: Improving Multimodal Understanding via Vision-Language Unified Autoregressive Paradigm (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
