Research map / Time series / Long-term forecasting
FACTS: A Factored State-Space Framework for World Modelling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FACTS, a recurrent factored state-space model with a permutable graph-structured memory and attention-based memory-input routing, evaluated on multivariate forecasting, object-centric video prediction, and graph traffic prediction.
Problem
Spatial-temporal world modelling with high-dimensional long sequences: Transformers and Mamba-style SSMs handle spatial structure and input permutations inefficiently, and rigid state structures struggle when input-to-factor relations change over time.
Approach
Treat the input features and state-space memory as sets of nodes, route inputs to latent factors via attention-based memory-input routing, and make the SSM parameters functions of the initial memory Z_0 and inputs so recurrence can be parallelised, yielding permutation-equivariant memory and permutation-invariant recurrence.
Key results
- FACTS achieves best MSE and MAE on 6 of 9 long-term forecasting datasets versus 8 TSLib baselines, per the paper's Table.
- On CLEVRER slot dynamics prediction FACTS reaches LPIPS 0.09, versus 0.11 for SlotFormer.
- On MOVi-A object discovery (video reconstruction) FACTS reaches FG-ARI 0.75 and mIoU 0.60, above OC-SlotSSM (0.68 / 0.55) and SAVi (0.64 / 0.43).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FACTS instead of the strongest listed baseline | Long-term MTS forecasting · ETTh2 | MSE: 0.381 -> 0.373 | Helped |
| Use FACTS instead of the strongest listed baseline | Long-term MTS forecasting · Electricity | MSE: 0.170 -> 0.166 | Helped |
| Use FACTS instead of the strongest listed baseline | Long-term MTS forecasting · Traffic | MSE: 0.414 -> 0.472 | Hurt |
| Use FACTS instead of the strongest listed baseline | Long-term MTS forecasting · ETTm2 | MSE: 0.281 -> 0.281 | No clear effect |
| Use FACTS for slot dynamics prediction | CLEVRER · CLEVRER | LPIPS: 0.11 -> 0.09 | Helped |
| Use FACTS for unsupervised object discovery | MOVi-A video reconstruction · MOVi-A | FG-ARI: 0.68 -> 0.75 | Helped |
| Use FACTS instead of TESTAM | METR-LA node prediction · METR-LA | MAPE: 9.67% -> 9.08% | Helped |
| Use MS-Conv2d embedder instead of Conv2d embedder (ablation) | FACTS · MTSF Electricity, prediction length 720 | MSE: 0.228 -> 0.198 | Helped |
| Use MS-Conv2d embedder instead of DFT embedder (ablation) | FACTS · MTSF Electricity, prediction length 720 | MSE: 0.236 -> 0.198 | Helped |
| Use Conv2d embedder instead of MS-Conv2d embedder (ablation) | FACTS · MTSF Electricity, prediction length 336 | MSE: 0.171 -> 0.170 | No clear effect |
| Use DFT embedder instead of MS-Conv2d embedder (ablation) | FACTS · MTSF Electricity, prediction length 336 | MSE: 0.171 -> 0.171 | No clear effect |
| Shuffle input feature order at test time | FACTS · MTS forecasting tasks | prediction performance | No clear effect |
Surprising findings
- Under shuffled input feature order, iTransformer and S-Mamba degrade substantially (e.g. S-Mamba's Traffic MSE more than triples) while FACTS stays stable.
Limitations
- Experiments are limited to relatively small-scale settings; scalability to larger models and datasets is untested.
- Number of factors k must be predefined before training.
- Ablations show modest sensitivity to embedder choice and factor count, and the paper does not establish that factors correspond to objects in general.
Experimental setup
Datasets: ETTm1 ETTm2 ETTh1 ETTh2 Electricity Exchange Traffic Weather Solar-Energy CLEVRER OBJ3D MOVi-A
Benchmarks: ETTm1 ETTm2 ETTh1 ETTh2 Electricity Exchange Traffic Weather Solar-Energy CLEVRER OBJ3D MOVi-A
Metrics: MSE MAE LPIPS FG-ARI mIoU MAPE 12-step MAE
Baselines: Autoformer FEDformer TimesNet PatchTST DLinear Crossformer iTransformer S-Mamba PredRNN VQFormer G-SWM SlotFormer
Builds on: Mamba Slot Attention SlotFormer SAVi TSLib
Closest papers by meaning
- Convolutional State Space Models for Long-Range Spatiotemporal Modeling (NeurIPS 2023 · similarity 0.87)
- Understanding and Mitigating Bottlenecks of State Space Models through the Lens of Recency and Over-smoothing (ICLR 2025 · similarity 0.86)
- StateSpaceDiffuser: Bringing Long Context to Diffusion World Models (NeurIPS 2025 · similarity 0.86)
- State Space Models on Temporal Graphs: A First-Principles Study (NeurIPS 2024 · similarity 0.85)
- Video World Models with Long-term Spatial Memory (NeurIPS 2025 · similarity 0.85)
- DyG-Mamba: Continuous State Space Modeling on Dynamic Graphs (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.