Research map / Architectures and efficiency / Attention and transformers
CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CirT, a Transformer for global subseasonal-to-seasonal (S2S) forecasting that uses latitude-based circular patches and Fourier-domain self-attention to respect the spherical geometry and periodicity of weather data, trained to directly predict weeks 3-4 and 5-6 averages on ERA5.
Problem
Subseasonal-to-seasonal weather forecasting is hard due to atmospheric chaos, and existing data-driven global models treat the spherical latitude-longitude grid as a planar image, distorting spatial relations and ignoring longitudinal periodicity.
Approach
Partition the graticule by latitude into equal-length-per-row circular patches as tokens, and apply DFT along each patch so that multi-head self-attention mixes patches in the frequency domain before an inverse DFT; the model directly predicts biweekly averages rather than iterating autoregressively.
Key results
- CirT achieves z500 RMSE of 477 m^2/s^2 at weeks 3-4 and 471 at weeks 5-6, versus 602 for the best baseline (ClimaX) at weeks 3-4.
- CirT attains t500 ACC of 0.988 at both weeks 3-4 and 5-6, outperforming all data-driven and numerical baselines in the reported tables.
- Ablation shows circular patching plus Fourier mixing reduces z500 RMSE from 497/494 (grid, FT) to 477/471 (circular, FT) at weeks 3-4/5-6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace grid patches with circular patches (ablation) | CirT without Fourier transform · ERA5 test, Weeks 3-4, z500 | RMSE: 516 to 502 | Helped |
| Replace grid patches with circular patches (ablation) | CirT with Fourier transform · ERA5 test, Weeks 3-4 and Weeks 5-6, z500 | RMSE: 497 to 477; 494 to 471 | Helped |
| Add Fourier transform while keeping grid patches (ablation) | CirT · ERA5 test, Weeks 3-4, z850 and t2m | RMSE: 319 to 324; 2.554 to 2.583 | Hurt |
| Add Fourier transform while keeping grid patches (ablation) | CirT · ERA5 test, Weeks 5-6, z850, t850 and t2m | RMSE: 319 to 320; 2.113 to 2.112; 2.578 to 2.650 | Hurt |
| Use direct S2S prediction instead of autoregressive next-day prediction (ablation) | CirT · ERA5 test, Weeks 3-4 and Weeks 5-6, z500 | RMSE: 781 to 477; 813 to 471 | Helped |
| Fine-tune embedding and decoder rather than directly train CirT (ablation) | CirT · ERA5 test, Weeks 3-4, z500 | RMSE: 477 to 480 | Hurt |
| Fine-tune embedding and decoder rather than directly train CirT (ablation) | CirT · ERA5 test, Weeks 3-4, t500 | RMSE: 1.687 to 1.660 | Helped |
| Fine-tune embedding and decoder rather than directly train CirT (ablation) | CirT · ERA5 test, Weeks 5-6, z500 | RMSE: 471 to 485 | Hurt |
| Fine-tune embedding and decoder rather than directly train CirT (ablation) | CirT · ERA5 test, Weeks 5-6, t850 | RMSE: 1.933 to 1.923 | Helped |
| Fine-tune decoder only rather than directly train CirT (ablation) | CirT · ERA5 test, Weeks 3-4, z500 | RMSE: 477 to 540 | Hurt |
| Fine-tune decoder only rather than directly train CirT (ablation) | CirT · ERA5 test, Weeks 5-6, z500 | RMSE: 471 to 588 | Hurt |
Surprising findings
- Applying the Fourier transform to grid patches does not consistently help and increases some errors (e.g., z850 and t2m), indicating patching design matters.
- Autoregressive CirT performs far worse than direct prediction (e.g., z500 RMSE 781 vs 477 at weeks 3-4).
Limitations
- Only 2D architecture; pressure levels are encoded into a single embedding, so vertical inductive biases are not modeled.
- Baselines like GraphCast could not be run for weeks 5-6 due to GPU memory; FourCastNetV2 lacks 2m temperature and Pangu-Weather inference uses pretrained weights without S2S fine-tuning, making comparisons partly unequal.
- Training is limited to 20 epochs with a single 1.5 degree resolution and a short training setup.
Experimental setup
Datasets: ERA5
Benchmarks: ERA5
Metrics: latitude-weighted RMSE ACC MS-SSIM
Baselines: FourCastNetV2 GraphCast PanguWeather ClimaX UKMO NCEP CMA ECMWF
Builds on: Vision Transformer FourCastNet Pangu-Weather ClimaX GraphCast
Closest papers by meaning
- ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction (NeurIPS 2024 · similarity 0.85)
- OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time Scales (NeurIPS 2025 · similarity 0.85)
- On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting (NeurIPS 2025 · similarity 0.84)
- Scaling transformer neural networks for skillful and reliable medium-range weather forecasting (NeurIPS 2024 · similarity 0.83)
- Probablistic Emulation of a Global Climate Model with Spherical DYffusion (NeurIPS 2024 · similarity 0.83)
- Mesh Interpolation Graph Network for Dynamic and Spatially Irregular Global Weather Forecasting (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.