Research map / Probabilistic and scientific modeling / Molecules and proteins
Factor Augmented Tensor-on-Tensor Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FATTNN, which extracts low-dimensional factor tensors from tensor covariates via a TIPUP-based tensor factor model and feeds them into a temporal convolutional network to forecast tensor responses.
Problem
Tensor-on-tensor time series forecasting, where both covariates and responses are multi-dimensional arrays and the relationship may be nonlinear, while preserving tensor structure and keeping computation low.
Approach
Integrate a Tucker-style tensor factor model (estimated by TIPUP and an iterative refinement) into a TCN: the network takes low-dimensional estimated factor tensors instead of raw covariate tensors, reducing dimensionality while keeping tensor structure and allowing nonlinear mapping.
Key results
- In simulations, FATTNN test MSE 6.353 vs TCN 9.812 in setting 1, and 511.9 vs 670.5 in setting 2, with lower computational time than TCN.
- On NYC taxi midtown, FATTNN MSE 7.399 vs TCN 12.87 and TRL 8.565, with about 3.4x speedup over TCN.
- On FMRI, Conv-TT-LSTM achieves lower MSE (0.03871) than FATTNN (0.06634), which the authors attribute to Conv-TT-LSTM being designed for image prediction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FATTNN instead of TCN | Tensor-on-tensor forecasting · Simulation 1 | Test MSE: 9.812 (3.13) -> 6.353 (2.52) | Helped |
| Use FATTNN instead of TCN | Tensor-on-tensor forecasting · Simulation 2 | Test MSE: 670.5 (25.8) -> 511.9 (22.5) | Helped |
| Use FATTNN instead of TCN | Tensor-on-tensor forecasting · Simulation 3 | Test MSE: 1074 (32.6) -> 696.7 (26.4) | Helped |
| Use FATTNN instead of TCN | FAO crop-to-crop forecasting · FAO crop ~ crop | Test MSE: 14.11 [10.83, 17.58] -> 7.437 [6.18, 11.44] | Helped |
| Use FATTNN instead of TCN | FAO crop-to-livestock forecasting · FAO crop ~ livestock | Test MSE: 32.53 [24.87, 36.10] -> 21.65 [19.56, 25.10] | Helped |
| Use FATTNN instead of TRL | NYC taxi forecasting · NYC taxi — midtown | Test MSE: 8.565 [7.89, 10.12] -> 7.399 [6.53, 8.03] | Helped |
| Use FATTNN instead of TCN | NYC taxi forecasting · NYC taxi — downtown | Test MSE: 14.58 [13.36, 17.42] -> 9.493 [8.46, 11.03] | Helped |
| Use FATTNN instead of Conv-TT-LSTM | FMRI image prediction · FMRI data | Test MSE: 0.03871 [0.0337, 0.0914] -> 0.06634 [0.053, 0.075] | Hurt |
| Use FATTNN instead of TCN | FMRI image prediction · FMRI data | Test MSE: 0.1236 [0.0997, 0.148] -> 0.06634 [0.053, 0.075] | Helped |
| Use factor-augmented FATTNN instead of raw-input TCN | Simulation tensor forecasting · Simulation 1 | Computational time: 01:22:02 -> 00:16:18 | Helped |
| Use factor-augmented FATTNN instead of raw-input TCN | FAO crop-to-crop forecasting · FAO crop ~ crop | Computational time: 00:55:33 -> 00:15:28 | Helped |
| Use factor-augmented FATTNN instead of raw-input TCN | NYC taxi forecasting · NYC taxi — downtown | Computational time: 03:25:55 -> 00:32:46 | Helped |
Surprising findings
- Conv-TT-LSTM, a specialized image-prediction method, beats FATTNN on the FMRI task in MSE.
- The baseline Conv-TT-LSTM substantially underperforms simpler methods on the FAO and NYC time-series tasks.
Limitations
- Designed primarily for time series; authors acknowledge TIPUP-TCN may not fully exploit non-temporal data and that higher-order tensors (K>4) may need Tensor-Train or Hierarchical Tucker.
- Loses to a specialized image-prediction model (Conv-TT-LSTM) on FMRI in MSE.
Experimental setup
Datasets: FAO crops and livestock data NYC taxi trip data Haxby FMRI dataset (via NiLearn) simulated tensor time series
Benchmarks: simulation studies (3 configurations) FAO crop and livestock prediction tasks NYC taxi midtown and downtown FMRI Haxby slice prediction
Metrics: test MSE computational time bootstrap confidence intervals
Baselines: TCN Multiway LSTM TRL Conv-TT-LSTM
Builds on: TIPUP temporal convolutional network multiway tensor-on-tensor regression tensor regression layer
Closest papers by meaning
- High-Dimensional Tensor Regression With Oracle Properties (ICML 2025 · similarity 0.89)
- Implicit Regularization in Tensor Factorization (ICML 2021 · similarity 0.88)
- Boosted Sparse and Low-Rank Tensor Regression (NeurIPS 2018 · similarity 0.88)
- Low-Rank Tensor Transitions (LoRT) for Transferable Tensor Regression (ICML 2025 · similarity 0.87)
- Geometry-aware training of factorized layers in tensor Tucker format (NeurIPS 2024 · similarity 0.87)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.