Research map / Probabilistic and scientific modeling / Molecules and proteins
Fixing the Double Penalty in Data-Driven Weather Forecasting Through a Modified Spherical Harmonic Loss Function
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AMSE, a parameter-free spectral modification of MSE that separates amplitude and decorrelation losses, and fine-tunes GraphCast with it to get sharper deterministic forecasts with higher effective resolution and better lagged-ensemble CRPS.
Problem
Data-driven deterministic weather forecasting models trained with MSE produce overly smooth forecasts because the "double penalty" causes the model to shrink unpredictable fine scales, reducing effective resolution and realistic variability.
Approach
Rewrites MSE in spherical harmonic space as an amplitude term plus a coherence term and replaces the spectral product with a max, yielding AMSE, a drop-in loss that rewards correct spectral amplitude and correlation without smoothing toward the mean.
Key results
- Effective resolution of the fine-tuned model increases from about 1,250 km (control, 5-day forecasts) to about 160 km (AMSE AR12).
- AMSE AR12 shows consistent CRPS improvements over the control for most variables and lead times in lagged ensemble verification while eRMSE changes little.
- AMSE AR12 retains HRES-like tropical cyclone intensity, removing the control's weak-intensity bias, with fine-tuning costing about 26.2 GPU-days on 4 A100s per node.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune GraphCast with AMSE for 12 autoregressive steps instead of the control training | GraphCast · Lagged ensemble, most variables and lead times | CRPS | Helped |
| Fine-tune GraphCast with AMSE for 12 autoregressive steps instead of the control training | GraphCast · Lagged ensemble, eRMSE | eRMSE | No clear effect |
| Fine-tune GraphCast with AMSE for 12 autoregressive steps instead of the control training | GraphCast · Fine-scale effective resolution | effective resolution: 1,250 km to 160 km | Helped |
| Fine-tune GraphCast with AMSE for 12 autoregressive steps | GraphCast · Tropical cyclone intensity, 20 June--19 September 2022 initializations | intensity error | Helped |
| Fine-tune GraphCast with AMSE for 12 autoregressive steps | GraphCast · Tropical cyclone forecasts, 20 June--19 September 2022 initializations | mean absolute position error | No clear effect |
| Fine-tune GraphCast with AMSE for 12 autoregressive steps | GraphCast · Surface wind extremes | wind-speed distribution realism | Helped |
| Fine-tune GraphCast with AMSE for 12 autoregressive steps | GraphCast · 2-meter temperature distribution | temperature distribution realism | No clear effect |
| Replace AMSE with MSE (ablation) | GraphCast fine-tuning for 12 autoregressive steps · Fine-scale forecast variability | spectral sharpness | Hurt |
| Replace AMSE with MAE (ablation) | GraphCast fine-tuning for 12 autoregressive steps · Medium scales, wavenumbers 32--200 at longer lead times | spectral sharpness | Helped |
| Replace AMSE with MAE (ablation) | GraphCast fine-tuning for 12 autoregressive steps · Lagged ensemble, headline variables and levels | CRPS | Helped |
| Train for one autoregressive step instead of 12 (ablation) | GraphCast fine-tuned with AMSE · Longer forecasts | forecast skill | Hurt |
| Train for one autoregressive step instead of 12 (ablation) | GraphCast fine-tuned with AMSE · Lagged ensemble | CRPS | Hurt |
Surprising findings
- Training with MAE also moderately improves sharpness over MSE, which the authors attribute to the median prediction being smoother than realizations at long lead times.
- Single-step AR1 fine-tuning gives greater ensemble spread but significantly worse CRPS due to lower skill, and AR1 and AR12 spread-error ratios converge at longer lead times.
Limitations
- Spherical harmonic decomposition is simple for global lat/lon grids but hard for local-area or irregular grids; alternatives like wavelets are only suggested.
- Smoothing persists at mesoscales for multi-step fine-tuning; precipitation does not benefit because its spectral decomposition is not natural.
- AMSE is not a proper metric (violates triangle inequality) and its out-of-distribution or climate behavior is untested.
Experimental setup
Datasets: HRES-fc0 (ECMWF HRES initial conditions 2016-2021) ERA5 accumulated precipitation IBTrACS
Benchmarks: calendar year 2022 HRES verification lagged ensemble CRPS/eRMSE/spread-error ratio tropical cyclone intensity and track (IBTrACS) station surface wind and temperature quantile-quantile
Metrics: amplitude ratio coherence CRPS eRMSE spread-error ratio mean maximum wind error mean absolute position error
Baselines: GraphCast (control) MSE AR12 MAE AR12 AMSE AR1 NeuralGCM HRES
Builds on: GraphCast MSE double penalty analysis NeuralGCM AdamW CRPS
Closest papers by meaning
- DiffDA: a Diffusion model for weather-scale Data Assimilation (ICML 2024 · similarity 0.86)
- Towards a Self-contained Data-driven Global Weather Forecasting Framework (ICML 2024 · similarity 0.85)
- ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction (NeurIPS 2024 · similarity 0.84)
- Fourier Amplitude and Correlation Loss: Beyond Using L2 Loss for Skillful Precipitation Nowcasting (NeurIPS 2024 · similarity 0.84)
- Tyche: One Step Flow for Efficient Probabilistic Weather Forecasting (NeurIPS 2026 · similarity 0.83)
- Generalizing Weather Forecast to Fine-grained Temporal Scales via Physics-AI Hybrid Modeling (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.