Research map / Time series / Foundation models for time series
PowerPM: Foundation Model for Power Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PowerPM, a ~250M-parameter foundation model for electricity time series pre-trained on hierarchical private ETS data with masked ETS modeling and dual-view contrastive learning, evaluated on private and four public hierarchical load datasets.
Problem
Learning generic representations of electricity time series (ETS) for diverse power-system tasks is hard because of hierarchical structure, exogenous-driven temporal dependencies, and heterogeneous consumption behavior across instances.
Approach
A temporal Transformer encoder with exogenous-variable embeddings feeds an R-GCN hierarchical encoder over a city-district-user-cluster graph, pre-trained with a mix of random and causal masked reconstruction plus a dual-view (temporal and instance) contrastive loss.
Key results
- On the private dataset, PowerPM achieves the best MSE on most forecasting tasks, e.g., 0.3378 vs 0.3923 for the best baseline on exclusive user 4-step forecasting, and 0.1907 average on solar generation forecasting with PatchTST-level baselines close behind.
- On public datasets, PowerPM achieves the lowest average MSE on CAISO state forecasting (0.3359 vs 0.3637 for PatchTST) and PJM state forecasting (0.3289 vs 0.3557 for the next best freeze variant).
- Model scale experiments report improvements with size across 30M, 70M, 120M and 250M variants; ablations show removing the hierarchical encoder causes the largest drop.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the hierarchical encoder (ablation) | PowerPM · 44 downstream tasks | task-specific evaluation metrics | Helped |
| Add dual-view contrastive learning (ablation) | PowerPM · Anomaly detection and classification tasks | F0.5 and accuracy | Helped |
| Use masked ETS modeling instead of vanilla random masking (ablation) | PowerPM · Forecasting tasks | MSE | Helped |
| Add the exogenous variables encoding module (ablation) | PowerPM · 44 downstream tasks | task-specific evaluation metrics | Helped |
| Increase model size from PowerPM-Tiny/Small/Medium to PowerPM | PowerPM · All downstream tasks | task-specific evaluation metrics | Helped |
| Fine-tune using a reduced proportion of downstream data | PowerPM · Downstream tasks | task-specific evaluation metrics | Hurt |
Surprising findings
- On the public NYISO and PJM datasets, the LLM-based Time-LLM is best on several horizons, so PowerPM is not uniformly superior.
Limitations
- Pre-training and most downstream data are private, limiting reproducibility.
- Some public-dataset results are not uniformly best (e.g., Time-LLM wins several NYISO and PJM horizons).
- Baselines are reproduced with modified epochs and learning rates; the comparison depends on these choices.
Experimental setup
Datasets: private hierarchical ETS pre-training data private downstream datasets (load forecasting, imputation, solar generation, electricity theft, clock anomaly, elderly a CAISO ISONE NYISO PJM
Benchmarks: private downstream tasks (44 tasks) CAISO ISONE NYISO PJM
Metrics: MSE MAE accuracy precision recall F0.5 F1
Baselines: GPT4TS Time-LLM UniTime PatchTST CoST TS2Vec TimesNet DLinear
Builds on: PatchTST TS2Vec CoST R-GCN Time-LLM
Closest papers by meaning
- Improving Day-Ahead Grid Carbon Intensity Forecasting by Joint Modeling of Local-Temporal and Cross-Variable Dependencies Across Different Frequencies (AAAI 2026 · similarity 0.84)
- PowerGraph: A power grid benchmark dataset for graph neural networks (NeurIPS 2024 · similarity 0.83)
- PF∆: A Benchmark Dataset for Power Flow under Load, Generation, and Topology Variations (NeurIPS 2025 · similarity 0.83)
- Timer: Generative Pre-trained Transformers Are Large Time Series Models (ICML 2024 · similarity 0.82)
- SynTSBench: Rethinking Temporal Pattern Learning in Deep Learning Models for Time Series (NeurIPS 2025 · similarity 0.82)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.