Research map / Video / Video segmentation and tracking
Convolutional Tensor-Train LSTM for Spatio-Temporal Learning
Jiahao Su, Wonmin Byeon, Jean Kossaifi, Furong Huang, Jan Kautz, Anima Anandkumar
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace ConvLSTM with Conv-TT-LSTM using sliding-window preprocessing | Moving-MNIST-2 video prediction · Moving-MNIST-2 (10 -> 30) | MSE: 33.08 -> 25.81 | Improved |
| Replace ConvLSTM with Conv-TT-LSTM using sliding-window preprocessing | Moving-MNIST-2 video prediction · Moving-MNIST-2 (10 -> 30) | SSIM: 0.806 -> 0.840 | Improved |
| Replace ConvLSTM with Conv-TT-LSTM using sliding-window preprocessing | Moving-MNIST-2 video prediction · Moving-MNIST-2 (10 -> 30) | LPIPS: 140.1 -> 90.38 | Improved |
| Use sliding-window rather than fixed-window preprocessing (ablation) | Conv-TT-LSTM · Moving-MNIST-2 (10 -> 30) | SSIM: 0.831 -> 0.840 | Improved |
| Use 4 layers, teacher forcing, and L1 loss (ablation) | Conv-TT-LSTM · Moving-MNIST-2 (10 -> 30) | MSE: 37.19 -> 31.46 | Improved |
| Use teacher forcing instead of scheduled sampling (ablation) | Conv-TT-LSTM · Moving-MNIST-2 (10 -> 30) | MSE: 33.08 -> 30.27 | Improved |
| Use L1 loss instead of L1 + L2 loss (ablation) | Conv-TT-LSTM · Moving-MNIST-2 (10 -> 30) | MSE: 36.95 -> 34.84 | Improved |
| Increase base filter size from 3 to 5 (ablation) | Conv-TT-LSTM · Hyperparameter search | performance | Improved |
| Increase tensor-train order (ablation) | Conv-TT-LSTM · Hyperparameter search | performance | Mixed |
| Increase tensor ranks or history steps beyond the selected values (ablation) | Conv-TT-LSTM · Hyperparameter search | performance | No clear effect |
| Remove convolutional operations by using 1 x 1 filters (ablation) | Conv-TT-LSTM · Ablation study (benchmark details unavailable) | performance | Hurt |
| Replace higher-order model with a first-order model (ablation) | Conv-TT-LSTM · Ablation study (benchmark details unavailable) | performance | Hurt |
Closest papers by meaning
- Lattice Long Short-Term Memory for Human Action Recognition (ICCV 2017 · similarity 0.87)
- Streaming Factor Trajectory Learning for Temporal Tensor Decomposition (NeurIPS 2023 · similarity 0.87)
- Factorized Higher-Order CNNs With an Application to Spatio-Temporal Emotion Estimation (CVPR 2020 · similarity 0.87)
- Tensor-Train Recurrent Neural Networks for Video Classification (ICML 2017 · similarity 0.87)
- Eidetic 3D LSTM: A Model for Video Prediction and Beyond (ICLR 2019 · similarity 0.87)
- Convolutional State Space Models for Long-Range Spatiotemporal Modeling (NeurIPS 2023 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
