Research map / Video / Action recognition
Recurrent Mixture Density Network for Spatiotemporal Visual Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LSTM temporal aggregation (vs RNN units) in the saliency model (ablation) | RMDN saliency predictor · Hollywood2 saliency | AUC: 0.8745 to 0.8866 (with 80 fixations/frame) | Helped |
| remove the recurrent link between consecutive time steps (ablation) | RMDN saliency predictor · Hollywood2 saliency | AUC: AUC 1.2% lower than RNN version and 2.4% lower than LSTM version | Hurt |
| increase fixations per frame during training (80 to 150) with LSTM(256) (ablation) | RMDN saliency predictor · Hollywood2 saliency | CC: 0.4606 to 0.6007 (LSTM 128/80 vs LSTM 256/150) | Helped |
| use RMDN saliency map (vs central-bias-only baseline) for saliency prediction | RMDN full · Hollywood2 saliency | AUC: 0.9037 vs 0.8725 for Trained Central Bias | Helped |
| use C3D fully-connected features instead of last convolutional features for saliency (ablation) | RMDN saliency predictor · Hollywood2 saliency | AUC: at least 1.5% lower AUC | Hurt |
| fine-tune C3D on Hollywood2 for action categorization (ablation) | RMDN saliency predictor · Hollywood2 saliency | AUC | No clear effect |
| use deep (multi-layer) LSTM instead of one-layer LSTM (ablation) | RMDN saliency predictor · Hollywood2 saliency | AUC | No clear effect |
| reduce GMM components from 20 to 2 (ablation) | RMDN saliency predictor · Hollywood2 saliency | AUC: AUC 0.8836 vs about 0.90 with C=20; NSS down 0.3 | Hurt |
| add saliency-weighted input stream concatenated with context C3D features | C3D FC6 action recognition · Hollywood2 mAP | mAP: 47.00% to 54.85% | Helped |
| use saliency to weight features in the feature mode instead of input pixels (ablation) | C3D CONV5 action recognition · Hollywood2 mAP | mAP: feature mode 45.62% concat vs clip mode 55.49% concat (ground truth saliency) | Hurt |
| use weighted input alone (without context concatenation) with predicted saliency (ablation) | C3D FC6 action recognition · Hollywood2 mAP | mAP: 39.00% alone vs 47.00% context-only | Hurt |
| use predicted saliency instead of ground-truth saliency for concatenated input (ablation) | C3D FC6 action recognition · Hollywood2 mAP | mAP: 55.12% GT vs 54.85% predicted | No clear effect |
Closest papers by meaning
- Revisiting Video Saliency: A Large-Scale Benchmark and a New Model (CVPR 2018 · similarity 0.90)
- Going From Image to Video Saliency: Augmenting Image Salience With Dynamic Attentional Push (CVPR 2018 · similarity 0.89)
- A Motion-aware Spatio-temporal Graph for Video Salient Object Ranking (NeurIPS 2024 · similarity 0.88)
- Attentional Push: A Deep Convolutional Network for Augmenting Image Salience With Shared Attention Modeling in Social Scenes (CVPR 2017 · similarity 0.88)
- Inferring Shared Attention in Social Scene Videos (CVPR 2018 · similarity 0.88)
- CaRDiff: Video Salient Object Ranking Chain of Thought Reasoning for Saliency Prediction with Diffusion (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.