Research map / Architectures and efficiency / Attention and transformers
Attention-Based Models for Snow-Water Equivalent Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies transformer-style spatial attention, temporal attention, and their average ensemble to predict daily snow-water equivalent at 323 SNOTEL stations in the Western U.S., outperforming LSTM and linear regression on NSE.
Problem
Predicting daily snow-water equivalent (SWE) at any location and date from sparse SNOTEL station data and high-resolution weather and satellite observables, where prior ML work largely ignored spatial correlations.
Approach
Reformulates SWE prediction as sequence-to-sequence over either locations (spatial attention, same day across stations) or days (temporal attention, one station across the season), using a transformer encoder with a simple average ensemble of the two.
Key results
- Ensemble attention places 82.04% of locations at NSE > 0.50, versus 65.63% for spatial attention and 62.22% for temporal attention.
- All three attention models outperform LSTM and linear regression in the NSE group distribution; temporal attention is only slightly better than LSTM.
- Spatial attention performs best at the lowest elevation quartile (median NSE 0.50), while temporal attention is best at the highest quartile (0.69).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine spatial and temporal attention predictions by averaging | Spatial_att and Temporal_att · Western U.S. SNOTEL test locations | locations with NSE > 0.50: 65.63% and 62.22% -> 82.04% | Helped |
| Use spatial attention instead of temporal attention | Attention-based SWE prediction · Lowest elevation quartile [0%, 25%) | median NSE: 0.24 -> 0.50 | Helped |
| Use ensemble attention instead of spatial attention | Attention-based SWE prediction · Elevation quartile [25%, 50%) | median NSE: 0.48 -> 0.70 | Helped |
| Use ensemble attention instead of spatial attention | Attention-based SWE prediction · Elevation quartile [50%, 75%) | median NSE: 0.61 -> 0.72 | Helped |
| Use temporal attention instead of spatial attention | Attention-based SWE prediction · Highest elevation quartile [75%, 100%) | median NSE: 0.52 -> 0.69 | Helped |
| Use temporal attention rather than LSTM sequence processing | SWE prediction · Western U.S. SNOTEL test locations | NSE | Helped |
Surprising findings
- Temporal attention performs only slightly better than LSTM, suggesting limited benefit from temporal attention alone in this setting.
- Spatial and temporal attention show complementary errors, with the simple average ensemble yielding the best overall results.
Limitations
- Evaluated only on 323 SNOTEL stations in the Western U.S. with a single test split of five years.
- Ensemble uses only a simple average; more sophisticated ensembling was left for future work.
- Forecasting and uncertainty quantification are not addressed; deployment to spatially complete maps is only a roadmap.
Experimental setup
Datasets: SNOTEL daily SWE, precipitation, and temperature (822 stations filtered to 323, 2002-2019) Passive microwave brightness temperature satellite observations (19VE, 37VE) Elevation, latitude/longitude, southness, and land cover static features
Benchmarks: SNOTEL SWE prediction on 323 stations, 5 held-out water years (2002-2019 period)
Metrics: Nash-Sutcliffe Efficiency (NSE) mean error in daily SWE mean error in annual maximum SWE
Baselines: LSTM Linear regression
Builds on: transformer LSTM
Closest papers by meaning
- Multi-Scale Graph Learning for Anti-Sparse Downscaling (AAAI 2025 · similarity 0.82)
- DroughtSet: Understanding Drought Through Spatial-Temporal Learning (AAAI 2025 · similarity 0.81)
- iTrendRNN: An Interpretable Trend-Aware RNN for Meteorological Spatiotemporal Prediction (AAAI 2024 · similarity 0.81)
- GLH-Water: A Large-Scale Dataset for Global Surface Water Detection in Large-Size Very-High-Resolution Satellite Imagery (AAAI 2024 · similarity 0.80)
- AlphaPre: Amplitude-Phase Disentanglement Model for Precipitation Nowcasting (CVPR 2025 · similarity 0.80)
- Physics-Guided Fair Graph Sampling for Water Temperature Prediction in River Networks (AAAI 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.