Research map / Probabilistic and scientific modeling / Variational and generative sampling
VITA: Variational Pretraining of Transformers for Climate-Robust Crop Yield Forecasting
Adib Hasan, Mardavij Roozbehani, Munther A. Dahleh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add variational pretraining objective (vs MSE reconstruction) with standard normal prior (ablation) | VITA transformer encoder · Corn Belt corn, 5 extreme years | R2: 0.660 -> 0.706 | Improved |
| use seasonality-aware sinusoidal prior instead of standard normal prior (ablation) | VITA · Corn Belt corn, 5 extreme years | R2: 0.706 -> 0.729 | Improved |
| use sinusoidal prior instead of standard normal prior (ablation) | VITA · Corn Belt soybean, 5 extreme years | R2: 0.698 -> 0.722 | Improved |
| add pretraining on weather grids excluding continental US (ablation) | VITA (Std. Normal prior) · Corn Belt corn, 27 hyperparameter configs | R2: 0.463 -> 0.632 | Improved |
| add full Americas pretraining (vs random init) (ablation) | VITA (Std. Normal prior) · Corn Belt corn, 27 hyperparameter configs | R2: 0.463 -> 0.706 | Improved |
| add full Americas pretraining (vs random init) (ablation) | VITA (Sinusoidal prior) · Corn Belt soybean, 27 hyperparameter configs | R2: 0.569 -> 0.698 | Improved |
| add pretraining (vs random init) to T-BERT (ablation) | T-BERT · Corn Belt corn and soybean | R2: +10.8% corn, +5.0% soybean | Improved |
| replace VITA with Chronos-Bolt-tiny (4.5x larger, general-purpose pretraining) | yield forecasting model · Corn Belt corn, 5 extreme years | R2: 0.729 -> 0.525 | Hurt |
| replace VITA with SimMTM temporal masking pretraining | yield forecasting model · Corn Belt corn, 5 extreme years | R2: 0.729 -> 0.642 | Hurt |
| add soil data to non-pretrained models (XGBoost, CNN-RNN, GNN-RNN) | baselines vs VITA without soil · Corn Belt corn, 5 extreme years | R2: GNN-RNN 0.564 vs VITA-Sinusoidal 0.729 without soil | No clear effect |
| truncate weather at end of July (week 30) (ablation) | VITA-Sinusoidal · Corn Belt soybean, extreme years | R2: 0.722 -> 0.560 | Hurt |
| truncate weather at end of July (week 30) (ablation) | VITA-Sinusoidal · Corn Belt corn, extreme years | R2: 0.729 -> 0.689 | Hurt |
Closest papers by meaning
- MMST-ViT: Climate Change-aware Crop Yield Prediction via Multi-Modal Spatial-Temporal Vision Transformer (ICCV 2023 · similarity 0.89)
- YieldSAT: A Multimodal Benchmark Dataset for High-Resolution Crop Yield Prediction (CVPR 2026 · similarity 0.88)
- DroughtSet: Understanding Drought Through Spatial-Temporal Learning (AAAI 2025 · similarity 0.83)
- CirT: Global Subseasonal-to-Seasonal Forecasting with Geometry-inspired Transformer (ICLR 2025 · similarity 0.83)
- OmniCast: A Masked Latent Diffusion Model for Weather Forecasting Across Time Scales (NeurIPS 2025 · similarity 0.83)
- On the Integration of Spatial-Temporal Knowledge: A Lightweight Approach to Atmospheric Time Series Forecasting (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
