Research map / Vision-language and multimodal / Visual question answering
Learning Models for Actions and Person-Object Interactions with Transfer to Question Answering
Arun Mallya, Svetlana Lazebnik
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add weighted loss (ablation) | Fusion-1 · HICO test | mAP: 33.6 -> 36.0 | Improved |
| add weighted loss (ablation) | Fusion-2 · HICO test | mAP: 33.8 -> 36.1 | Improved |
| fuse person box and full image features early (Fusion-2) instead of full image only | VGG-16 · HICO test | mAP: 29.4 -> 33.8 | Improved |
| fuse features early (Fusion-2) instead of averaging independent predictions (Scene-RCNN) | VGG-16 · HICO test | mAP: 29.0 -> 33.8 | Improved |
| use person bounding box only instead of adding full image (ablation) | VGG-16 with MIL · HICO test | mAP: 29.4 -> 14.6 | Hurt |
| Fusion-2 channel reduction before stacking versus Fusion-1 | Fusion · HICO test | mAP: 33.6 -> 33.8 | Improved |
| replace R*CNN secondary box search with full image box (Scene-RCNN) | VGG-16 · HICO test | mAP: 28.5 -> 29.0 | Improved |
| swap AlexNet for VGG-16 backbone with full image | full-image classifier · HICO test | mAP: 19.4 -> 29.4 | Improved |
| train with MIL over ground truth instances instead of a label per instance (ablation) | Fusion-2 · MPII test | mAP: 32.24 -> 31.89 | Hurt |
| use detected instead of ground truth person boxes (ablation) | Fusion-2 · MPII test | mAP: 32.24 -> 32.02 | No clear effect |
| replace ImageNet VGG-16 fc7 with HICO full-image network fc7 | MadLibs CCA · MadLibs Easy Person's Activity | accuracy: 80.79 -> 86.54 | Improved |
| use Fusion-2 on person boxes instead of HICO full-image fc7 | MadLibs CCA · MadLibs Person's Activity (Filtered Hard) | accuracy: 72.39 -> 73.46 | Improved |
Closest papers by meaning
- Graph-Structured Representations for Visual Question Answering (CVPR 2017 · similarity 0.87)
- Ask Your Neurons: A Deep Learning Approach to Visual Question Answering (IJCV 2017 · similarity 0.87)
- Combining Multiple Cues for Visual Madlibs Question Answering (IJCV 2018 · similarity 0.87)
- Multi-Question Learning for Visual Question Answering (AAAI 2020 · similarity 0.87)
- An Analysis of Visual Question Answering Algorithms (ICCV 2017 · similarity 0.87)
- A Dataset and Exploration of Models for Understanding Video Data Through Fill-In-The-Blank Question-Answering (CVPR 2017 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
