Research map / Video / Video segmentation and tracking
Improving Spatiotemporal Self-Supervision by Deep Reinforcement Learning
Uta Buchler, Biagio Brattoli, Bjorn Ommer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RL-selected permutation policy to the spatial task (ablation) | Spatial permutation task · Pascal VOC07 classification | mAP: 67.6 -> 71.3 | Improved |
| add RL-selected permutation policy to the temporal task (ablation) | Temporal permutation task · Pascal VOC07 classification | mAP: 64.1 -> 65.9 | Improved |
| add RL policy to the combined spatiotemporal model (ablation) | S+T · Pascal VOC07 classification | mAP: 72.0 -> 74.2 | Improved |
| add RL policy to the spatial task (ablation) | Spatial permutation task · UCF-101 action recognition | accuracy: 51.5 -> 54.6 | Improved |
| add RL policy to the temporal task (ablation) | Temporal permutation task · UCF-101 action recognition | accuracy: 52.8 -> 55.7 | Improved |
| add RL policy to the combined spatiotemporal model (ablation) | S+T · UCF-101 action recognition | accuracy: 57.3 -> 58.6 | Improved |
| train spatial and temporal tasks jointly instead of the spatial task alone (ablation) | Spatial task · Pascal VOC07 classification | mAP: 67.6 -> 72.0 | Improved |
| train spatial and temporal tasks jointly instead of the temporal task alone (ablation) | Temporal task · Pascal VOC07 classification | mAP: 64.1 -> 72.0 | Improved |
| train tasks in parallel instead of serially (spatial then temporal) (ablation) | S&T vs S+T · Pascal VOC07 classification | mAP: 69.8 -> 72.0 | Improved |
| train tasks in parallel instead of serially (ablation) | S&T vs S+T · UCF-101 action recognition | accuracy: 54.2 -> 57.3 | Improved |
| use trained RL policy instead of random permutation selection (ablation) | Self-supervised network · Validation accuracy after one epoch | relative accuracy to random policy: (125\pm14)\% | Improved |
| use inverse policy (discarded permutations) instead of random selection (ablation) | Self-supervised network · Validation accuracy after one epoch | relative accuracy to random policy: (78\pm16)\% | Hurt |
Closest papers by meaning
- Scaling and Benchmarking Self-Supervised Visual Representation Learning (ICCV 2019 · similarity 0.87)
- Revisiting Self-Supervised Visual Representation Learning (CVPR 2019 · similarity 0.86)
- Self-Supervised Spatiotemporal Learning via Video Clip Order Prediction (CVPR 2019 · similarity 0.86)
- A critical analysis of self-supervision, or what we can learn from a single image (ICLR 2020 · similarity 0.86)
- Self-Supervised Learning of Pose Embeddings From Spatiotemporal Relations in Videos (ICCV 2017 · similarity 0.85)
- A Theoretical Analysis of Self-Supervised Learning for Vision Transformers (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
