AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Omni-Weather: Unified Multimodal Foundation Model for Weather Generation and Understanding

Zhiwang Zhou, Yuandong Pu, Xuming He, Yidi Liu, Yixin Chen, Junchao Gong, Xiang Zhuang, Wanghan Xu, Qinglong Cao, SHIXIANG TANG, Yihao Liu, Wenlong Zhang and 1 more

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Omni-Weather is a unified multimodal foundation model that performs both weather generation and weather understanding within a single architecture, using a radar encoder for generation tasks.

Problem

Weather modeling needs both accurate prediction and mechanistic interpretation, but existing methods treat weather generation and understanding separately.

Approach

Unifies weather generation and understanding in one multimodal foundation model by integrating a radar encoder for generation tasks alongside a multimodal backbone for understanding.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train generation and understanding jointly rather than generation only (ablation)Bagel-7B-MoT · SEVIR nowcastingCSI-Mean: 0.303 -> 0.338Improved
train generation and understanding jointly rather than generation only (ablation)Bagel-7B-MoT · SEVIR inversionRMSE: 0.590 -> 0.514Improved
train jointly rather than understanding only (ablation)Bagel-7B-MoT · RadarQA frame understandingaccuracy: 81.95 -> 86.65Improved
train jointly rather than understanding only (ablation)Bagel-7B-MoT · RadarQA sequence understandingaccuracy: 54.34 -> 59.58Improved
add thinking inference to CoT-finetuned model (ablation)Omni-Weather · SEVIR nowcastingLPIPS: 0.182 -> 0.163Improved
add thinking inference to CoT-finetuned model (ablation)Omni-Weather · SEVIR nowcastingCSI-Mean: 0.347 -> 0.335Hurt
use thinking inference without CoT finetuning (ablation)Omni-Weather · SEVIR nowcastingCSI-Mean: 0.347 -> 0.237Hurt
add general MetaQuery data to SEVIR finetuning (ablation)Bagel-7B-MoT · SEVIR nowcastingCSI-Mean: 0.3352 -> 0.3471Improved
add general MetaQuery data to SEVIR finetuning (ablation)Bagel-7B-MoT · SEVIR nowcastingCRPS: 0.0231 -> 0.0226Improved
unified Omni-Weather vs specialized nowcasting baselinesOmni-Weather · SEVIR nowcastingCRPS: 0.031 -> 0.026Improved
unified model vs RadarQA on image understanding miss accuracyOmni-Weather · RadarQA image understandingMiss accuracy: 67.67 -> 92.21Improved
unified model vs RadarQA on sequence understandingOmni-Weather · RadarQA sequence understandingOverall accuracy: 66.17 -> 61.79Mixed

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 2 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy