Research map / Architectures and efficiency / RNNs and normalization
Long-Term Recurrent Convolutional Networks for Visual Recognition and Description
Jeff Donahue, Lisa Anne Hendricks, Marcus Rohrbach, Subhashini Venugopalan, Sergio Guadarrama, Kate Saenko, Trevor Darrell
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LRCN LSTM on top of per-frame CNN (flow input) | single-frame CNN · UCF101 | accuracy: 74.37 -> 77.28 | Improved |
| combine RGB and flow with weights 1/3, 2/3 | LRCN-fc6 · UCF101 | accuracy: 68.20 -> 82.34 | Improved |
| use LSTM instead of vanilla RNN (ablation) | LRCN_2f CaffeNet · COCO val image-to-caption retrieval | R@1: 30.2 -> 33.8 | Improved |
| use LSTM instead of vanilla RNN (ablation) | LRCN_2f CaffeNet · COCO val captioning | CIDEr-D: 0.703 -> 0.720 | Improved |
| factor the 2-layer LSTM (feed image only to layer 2) (ablation) | LRCN 2-layer CaffeNet · COCO val caption-to-image retrieval | R@1: 23.4 -> 25.0 | Improved |
| stack a second LSTM layer (unfactored vs 1 layer) (ablation) | LRCN CaffeNet · COCO val caption-to-image retrieval | R@1: 25.2 -> 23.4 | Hurt |
| fine-tune the CNN (ablation) | LRCN_2f VGGNet · COCO val image-to-caption retrieval | R@1: 46.0 -> 53.3 | Improved |
| fine-tune the CNN (ablation) | LRCN_2f VGGNet · COCO val captioning | CIDEr-D: 0.828 -> 0.896 | Improved |
| switch CaffeNet to VGGNet (ablation) | LRCN_2f fine-tuned · COCO val captioning | CIDEr-D: 0.764 -> 0.896 | Improved |
| sample best of N=100 (T=1.5) instead of greedy (ablation) | LRCN_2f VGGNet FT · COCO val captioning | CIDEr-D: 0.839 -> 0.896 | Improved |
| use beam width 3 instead of 1 (ablation) | LRCN_2f VGGNet FT · COCO val captioning | CIDEr-D: 0.839 -> 0.888 | Improved |
| increase beam width from 3 to 10 (ablation) | LRCN_2f VGGNet FT · COCO val captioning | CIDEr-D: 0.888 -> 0.886 | No clear effect |
Closest papers by meaning
- Making Convolutional Networks Recurrent for Visual Sequence Learning (CVPR 2018 · similarity 0.91)
- Beyond Temporal Pooling: Recurrence and Temporal Convolutions for Gesture Recognition in Video (IJCV 2016 · similarity 0.90)
- Task-Driven Convolutional Recurrent Models of the Visual System (NeurIPS 2018 · similarity 0.90)
- Learning Implicitly Recurrent CNNs Through Parameter Sharing (ICLR 2019 · similarity 0.89)
- Long-Term Temporal Convolutions for Action Recognition (TPAMI 2017 · similarity 0.89)
- Contextual Convolutional Networks (ICLR 2023 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
