Research map / Video / Action recognition
Controllable Attention for Structured Layered Video Decomposition
Jean-Baptiste Alayrac, Joao Carreira, Relja Arandjelovic, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add encoder gating masks (ablation) | C2 · Kinetics-600 val blended (transparency) | validation loss: 0.131 -> 0.120 | Improved |
| add encoder gating masks (ablation) | C2 · Kinetics-600 val blended (occlusion) | validation loss: 0.200 -> 0.190 | Improved |
| add the composition module (layers and composing coefficients) to the centrifuge (ablation) | Centrifuge · Kinetics-600 val blended (occlusion) | validation loss: 0.253 -> 0.200 | Improved |
| add the composition module (layers and composing coefficients) to the centrifuge (ablation) | Centrifuge · Kinetics-600 val blended (transparency) | validation loss: 0.149 -> 0.131 | Improved |
| use C2 instead of the twice as large predictor-corrector centrifuge | layer decomposition · Kinetics-600 val blended (occlusion) | validation loss: 0.264 -> 0.190 | Improved |
| use internal prediction control instead of deterministic control (ablation) | C3 · Kinetics-600 val blended (transparency) | validation loss: 0.191 -> 0.119 | Improved |
| use internal prediction control instead of deterministic control (ablation) | C3 · Kinetics-600 val blended (transparency) | control accuracy (%): 79.1% -> 77.7% | No clear effect |
| add audio control to C2 with deterministic control | C2 · Kinetics-600 val blended (transparency) | validation loss: 0.120 -> 0.191 | Hurt |
| decompose blended video with C2 before I3D classification (transparency) | I3D · Kinetics-600 val action recognition (transparency blended) | accuracy: 22.1 -> 40.1 | Improved |
| decompose blended video with C2 before I3D classification (occlusion) | I3D · Kinetics-600 val action recognition (occlusion blended) | top-1 accuracy: 21.3 -> 24.7 | Improved |
| use C2 instead of CentrifugePC to unblend before I3D (occlusion) | I3D pipeline · Kinetics-600 val action recognition (occlusion blended) | top-1 accuracy: 21.5 -> 24.7 | Improved |
| shift the control audio so it does not overlap the video (ablation) | C3 with internal prediction · Kinetics-600 val blended (transparency) | control accuracy (points): decreases by 1.4 percentage points | Hurt |
Closest papers by meaning
- Generative Omnimatte: Learning to Decompose Video into Layers (CVPR 2025 · similarity 0.85)
- Associating Objects and Their Effects in Video through Coordination Games (NeurIPS 2022 · similarity 0.85)
- HyperNVD: Accelerating Neural Video Decomposition via Hypernetworks (CVPR 2025 · similarity 0.85)
- Referring Layer Decomposition (ICLR 2026 · similarity 0.84)
- The Visual Centrifuge: Model-Free Layered Video Representations (CVPR 2019 · similarity 0.84)
- Language-guided Image Reflection Separation (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
