Research map / Time series / Causal and generative time series
TACTiS-2: Better, Faster, Simpler Attentional Copulas for Multivariate Time Series
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TACTiS-2, a transformer-based attentional copula model trained with a two-stage curriculum (marginals first, then copula) that gives linear-in-d parameters, faster convergence, and state-of-the-art multivariate probabilistic forecasting and interpolation.
Problem
Learning valid nonparametric copulas for multivariate probabilistic time series forecasting and interpolation; the permutation-based TACTiS objective needs O(d!) distributional parameters, giving poor training dynamics.
Approach
Replace the permutation-invariant copula objective with a two-stage maximum-likelihood curriculum (learn marginals with an independent copula, then freeze marginals and learn the attentional copula) using a dual encoder, which yields valid copulas with O(d) parameters.
Key results
- TACTiS-2 achieves the lowest mean CRPS-Sum on 4 of 5 Monash datasets (e.g., electricity 0.020 vs 0.021 for TACTiS; solar-10min 0.240 vs 0.311) and the best average rank of 1.9.
- TACTiS-2 reaches much lower NLL with fewer training FLOPs than TACTiS (e.g., fred-md NLL 0.378 vs 1.364 at 0.738e16 vs 1.956e16 FLOPs).
- On interpolation, TACTiS-2 achieves lower NLL than TACTiS on all five datasets (e.g., solar-10min -3.262 vs -0.948; kdd-cup 0.138 vs 1.050).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the permutation-based copula objective with TACTiS-2's two-stage copula training | TACTiS · Forecasting across electricity, fred-md, kdd-cup, solar-10min, and traffic | CRPS-Sum: electricity: 0.021 ± 0.005 -> 0.020 ± 0.005; fred-md: 0.042 ± 0.009 -> 0.035 ± 0.005; kdd-cup: 0.237 ± 0.013 -> 0.234 ± | Mixed |
| Replace the permutation-based copula objective with TACTiS-2's two-stage copula training | TACTiS · Forecasting across five benchmark datasets | NLL: 11.028 ± 3.616 -> 10.674 ± 2.867; 1.364 ± 0.253 -> 0.378 ± 0.076; 2.281 ± 0.770 -> 1.055 ± 0.713; $-$2.572 ± 0.093 -> $- | Helped |
| Replace the permutation-based copula objective with TACTiS-2's two-stage copula training | TACTiS · Forecasting across five benchmark datasets | FLOPs (× 10^16): 1.931 ± 0.182 -> 0.623 ± 0.018; 1.956 ± 0.192 -> 0.738 ± 0.022; 1.952 ± 0.208 -> 0.324 ± 0.014; 0.174 ± 0.018 -> ~0.078 | Helped |
| Replace the permutation-based copula objective with TACTiS-2's two-stage copula training | TACTiS · Interpolation across electricity, fred-md, kdd-cup, solar-10min, and traffic | NLL: $-$0.159 ± 0.007 -> $-$0.189 ± 0.034; $-$0.119 ± 0.021 -> $-$0.250 ± 0.017; 1.050 ± 0.045 -> 0.138 ± 0.043; $-$0.948 ± 0 | Helped |
| Remove the two-stage curriculum and train all model parameters jointly in one stage (ablation) | TACTiS-2 · kdd-cup and solar-10min forecasting | validation NLL | Hurt |
Surprising findings
- Without the two-stage curriculum (single joint training), the ablation reaches only slightly better NLL than TACTiS but needs more FLOPs, showing the curriculum is crucial.
- TACTiS slightly outperforms TACTiS-2 on traffic in CRPS-Sum (0.071 vs 0.078).
Limitations
- Validity of copulas holds only in the limit of infinite data and capacity; finite-sample convergence is not guaranteed.
- Restricted to continuous data because marginals use normalizing flows (DSF).
- Hyperparameter search with Optuna and backtesting is computationally costly; baselines' numbers taken from prior work.
Experimental setup
Datasets: electricity fred-md kdd-cup solar-10min traffic
Benchmarks: electricity fred-md kdd-cup solar-10min traffic
Metrics: CRPS-Sum CRPS Energy score NLL FLOPs average rank
Baselines: Auto-ARIMA ETS TempFlow SPD TimeGrad GPVar TACTiS
Builds on: TACTiS Deep Sigmoidal Flows Sklar's theorem GPVar
Closest papers by meaning
- Timer-XL: Long-Context Transformers for Unified Time Series Forecasting (ICLR 2025 · similarity 0.87)
- Revisiting Attention for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
- A Closer Look at Transformers for Time Series Forecasting: Understanding Why They Work and Where They Struggle (ICML 2025 · similarity 0.87)
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.87)
- TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables (NeurIPS 2024 · similarity 0.86)
- CATS: Enhancing Multivariate Time Series Forecasting by Constructing Auxiliary Time Series as Exogenous Variables (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.