Research map / Vision-language and multimodal / Human and embodied modeling
Jointly Modeling Embedding and Translation to Bridge Video and Language
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add joint visual-semantic relevance modeling to LSTM | LSTM video description model · YouTube2Text test | BLEU@4: 31.2 -> 45.3 | Helped |
| Add joint visual-semantic relevance modeling to LSTM | LSTM video description model · YouTube2Text test | METEOR: 26.9 -> 31.0 | Helped |
| Use VGG+C3D features rather than AlexNet features (ablation) | LSTM-E · YouTube2Text test | BLEU@4: 38.9 -> 45.3 | Helped |
| Combine VGG and C3D features rather than using VGG alone (ablation) | LSTM-E · YouTube2Text test | BLEU@4: 40.2 -> 45.3 | Helped |
| Combine VGG and C3D features rather than using C3D alone (ablation) | LSTM-E · YouTube2Text test | BLEU@4: 41.7 -> 45.3 | Helped |
| Increase LSTM hidden size from 128 to 512 (ablation) | LSTM-E (VGG+C3D) · YouTube2Text test | BLEU@4: 38.4 -> 45.3 | Helped |
| Increase LSTM hidden size from 128 to 512 (ablation) | LSTM-E (VGG+C3D) · YouTube2Text test | METEOR: 29.0 -> 31.0 | Helped |
| Set the coherence-relevance tradeoff parameter to about 0.7 (ablation) | LSTM-E · YouTube2Text test | BLEU and METEOR | Helped |
| Feed visual content only at the initial LSTM step rather than at every timestep (ablation) | LSTM-E · Video description generation | generation performance | Helped |
Closest papers by meaning
- End-to-End Convolutional Semantic Embeddings (CVPR 2018 · similarity 0.90)
- Video Captioning With Transferred Semantic Attributes (CVPR 2017 · similarity 0.89)
- Fully Convolutional Video Captioning with Coarse-to-Fine and Inherited Attention (AAAI 2019 · similarity 0.88)
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.87)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.87)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.