AI Paper Atlas

Research map / Vision-language and multimodal / Cross-modal learning

OmniScale: Scaling Any Modality Model Training with Model-Centric Distributed Recipe Zoo

Qianli Ma, Yaowei Zheng, Zhelun Shi, Zhongkai Zhao, Bin Jia, Ziyue Huang, Zhiqi Lin, Youjie Li, Jiacheng Yang, Yanghua Peng, Zhi Zhang, Xin Liu

AAAI 2026 · Vision-language and multimodal › Cross-modal learning · approach: Multimodal fusion with missing modalities · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use VeOmni instead of TorchTitanFSDP training · Qwen2-7B FSDP 8k, 128 GPUsthroughput: 6,639 -> 6,940Improved
use VeOmni instead of TorchTitanFSDP+SP4 · Qwen2-7B 32k, 128 GPUsthroughput: 3,727 -> 4,629Improved
use VeOmni instead of TorchTitanFSDP+SP8 · Qwen2.5-32B 32k, 128 GPUsthroughput: 821 -> 1,115Improved
use VeOmni instead of TorchTitanFSDP+SP4 · Qwen2-72B 32k, 128 GPUsthroughput: 497 -> 592Improved
use VeOmni instead of TorchTitanFSDP · Qwen2-7B 32k, 128 GPUsMFU (%): 39.68 -> 44.42Improved
use VeOmni instead of TorchTitanFSDP · Qwen2-7B 32k, 128 GPUsmemory (GB): 71.81 -> 31.41Improved
use VeOmni instead of TorchTitanFSDP · Qwen2-7B 64k, 128 GPUstrainability: OOM -> 3,725Improved
use VeOmni instead of TorchTitanFSDP+SP4 · Qwen2-72B 8k, 128 GPUsmemory (GB): 54.08 -> 58.62Hurt
use VeOmni instead of TorchTitanFSDP+SP8 · Qwen2-72B 128k, 128 GPUstrainability: OOM -> 303Improved
increase sequence parallel size from 4 to 8 (ablation)VeOmni · Qwen2-72B 128k, 128 GPUstrainability: OOM -> 303Improved
increase sequence length from 8k to 128k with SP4 (ablation)VeOmni · Qwen3-30B-A3B EP8, 128 GPUsMFU (%): 3.02 -> 17.92Improved
add SP4 on top of FSDP+EP8 at 8k (ablation)VeOmni · Qwen3-30B-A3B 8k, 128 GPUsthroughput: 2,776 -> 1,216Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy