Research map / Federated learning and privacy / Personalized and heterogeneous FL
Personalized Adapter for Large Meteorology Model on Devices: Towards Weather Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LM-Weather, which tames a pre-trained language model (GPT-2) with lightweight personalized adapters and shares only LoRA low-rank matrices in federated learning for on-device meteorological forecasting and imputation.
Problem
On-device meteorological variable modeling (forecasting and imputation) across heterogeneous weather stations under data heterogeneity, limited data, and communication constraints.
Approach
Plug a personalized adapter (trend/seasonal/residual decomposition-based task adapter plus FFN parameter adapter) into a frozen PLM with LoRA on query/value; only LoRA matrices are communicated in federated averaging while adapters stay local.
Key results
- Forecasting (multivariate-to-multivariate): LM-Weather averages 9.8% lower MAE than FL-GPT4TS and achieves best results in 29 of 40 MAE/RMSE settings.
- Imputation (50% mask): LM-Weather outperforms FL-GPT4TS by 5.7% and is best in 30 settings.
- Communication: transmits about 3.7% of trainable parameters (0.38M of 10.38M), with communication overhead 35.9% of FL-DLinear and 22.6% of FL-LightTS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use personalized LM-Weather instead of LM-Weather-ave | GPT2 weather modeling · ODW1T forecasting (Avg.) | MAE: 47.5 -> 45.4 | Helped |
| use personalized LM-Weather instead of FL-GPT4TS | on-device weather forecasting · ODW1T forecasting (Avg.) | MAE: 49.9 -> 45.4 | Helped |
| use personalized LM-Weather instead of FL-GPT4TS | on-device weather imputation · ODW1T imputation (Avg., 50% masking) | MAE: 25.1 -> 23.1 | Helped |
| add weather decomposition (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 50.8 -> 45.4 | Helped |
| add the trend component (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 50.9 -> 45.4 | Helped |
| add the seasonal component (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 50.1 -> 45.4 | Helped |
| add the residual component (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 49.3 -> 45.4 | Helped |
| add the prompt generator (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 53.8 -> 45.4 | Helped |
| share only low-rank matrices rather than other trainable parameters (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 49.4 -> 45.4 | Helped |
| use LoRA instead of full attention-parameter tuning (ablation) | LM-Weather · ODW1T forecasting (Avg.) | MAE: 43.2 -> 45.4; 42.7 -> 45.4 | Hurt |
| use LoRA instead of full attention-parameter tuning (ablation) | LM-Weather · ODW1T imputation (50% masking) | MAE: 22.4 -> 23.1; 22.2 -> 23.1 | Hurt |
Surprising findings
- Full fine-tuning of attention parameters instead of LoRA gives only a slight gain while costing over four times the parameters and much higher communication.
Limitations
- Relies on linear interpolation and manual correction for missing values and outliers in the datasets, which may bias evaluation.
- Zero-shot and few-shot results are reported only in limited settings; some baselines lack results at longer horizons due to insufficient data.
Experimental setup
Datasets: ODW1T ODW1V ODW2T ODW2V
Benchmarks: ODW1T ODW1V ODW2T ODW2V
Metrics: MAE RMSE
Baselines: FL-GPT4TS FL-Transformer FL-Informer FL-Reformer FL-Pyraformer FL-iTransformer FL-PatchTST FL-DLinear FL-LightTS LM-Weather-ave
Builds on: GPT4TS (one fits all) LoRA FedAvg PatchTST channel-independent patching
Closest papers by meaning
- WeatherGFM: Learning a Weather Generalist Foundation Model via In-context Learning (ICLR 2025 · similarity 0.87)
- WeatherSyn: An Instruction Tuning MLLM For Weather Forecasting Report Generation (ICML 2026 · similarity 0.87)
- EWMoE: An Effective Model for Global Weather Forecasting with Mixture-of-Experts (AAAI 2025 · similarity 0.85)
- Time-FFM: Towards LM-Empowered Federated Foundation Model for Time Series Forecasting (NeurIPS 2024 · similarity 0.85)
- AtmosSci-Bench: Evaluating the Recent Advance of Large Language Model for Atmospheric Science (NeurIPS 2025 · similarity 0.84)
- Time-LLM: Time Series Forecasting by Reprogramming Large Language Models (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.