AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

UNIVERSAL AND EFFICIENT LOADING BALANCING FOR RL TRAINING OF LARGE MULTIMODAL MODELS

Zerui Wang, Qinghao Hu, Chang Chen, Jiecheng Zhou, Haojie Duanmu, Xingcheng Zhang, Peng Sun, Dahua Lin

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Addresses system-level inefficiencies in RL training of vision-language models caused by heterogeneous multimodal data, proposing a load-balancing approach for the RL pipeline.

Problem

RL alignment of large VLMs is hampered by system bottlenecks (centralized data loading I/O, inference rollouts, model updates) due to extreme heterogeneity of multimodal data.

Approach

Load balancing across the RL training pipeline to handle heterogeneous multimodal data efficiently; specific mechanism not stated in the available abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add ShadowLoader (distributed metadata-driven data loading) (ablation)veRL · Image-Heavy, MiMo-VL-7B-RL, 128 H800 GPUsdata loading time: 375.0 s -> 3.9 sImproved
add ShadowLoader (ablation)veRL · Image-Heavy, MiMo-VL-7B-RL, 128 H800 GPUsstep time: 656.2 s -> 122.6 sImproved
add FlexUlysses alone without ShadowLoader (ablation)veRL · Image-Heavy, MiMo-VL-7B-RL, 128 H800 GPUsend-to-end throughput: 1.00x -> 1.17xImproved
add FlexUlysses on top of ShadowLoader (ablation)veRL+ShadowLoader · Image-Heavy, MiMo-VL-7B-RL, 128 H800 GPUsstep time: 122.6 s -> 85.7 sImproved
use FlexUlysses adaptive sharding instead of fixed-degree Ulysses-SP (SP=8)FlexRL · Image-Heavy, 7B, 32 GPUsend-to-end throughput speedup vs veRL: 1.13x -> 1.50xImproved
increase fixed Ulysses SP degree (ablation)Ulysses-SP · Image-Heavy, 7B, 32 GPUsattention balance ratio: 6.70 -> 1.12 (SP=4)Improved
FlexUlysses sharding for rollout and training when data loading is not the bottleneck (ablation)veRL+ShadowLoader · Image-Heavy, MiMo-VL-7B-RL, 128 H800 GPUsupdate actor time: 24.3 s -> 9.5 sImproved
switch from Ulysses-SP to FlexRL on larger models and video-heavy dataFlexRL · Only-Video, 7B, 128 GPUsthroughput: 50.67 -> 405.44Improved
generation (rollout) stage with FlexUlysses added to ShadowLoader (ablation)veRL+ShadowLoader · Image-Heavy, MiMo-VL-7B-RL, 128 H800 GPUsgeneration time: 64.1 s -> 64.4 sNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy