AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

ORION: Decoupling and Alignment for Unified Autoregressive Understanding and Generation

Taihang Hu, Mengting Chen, Jinsong Lan, Xiaoyong Zhu, Kaifu Zhang, Ming-Ming Cheng, Bo Zheng, Yaxing Wang

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ORION, a unified autoregressive multimodal LLM that decouples and aligns understanding and generation to resolve the semantic-structural conflict in monolithic architectures.

Problem

Monolithic autoregressive unified multimodal LLMs suffer a semantic-structural conflict: optimizing for low-level reconstructability in generation causes catastrophic forgetting of high-level semantic understanding.

Approach

Decouples the understanding and generation pathways within a unified autoregressive MLLM and aligns them, rather than using a single monolithic objective, so generation's reconstruction pressure does not erode semantic understanding.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add representation consistency loss L_KL (ablation)ORION (MLP head, 20M warm-up) · MMBenchaccuracy: 79.8 -> 83.7Improved
add representation consistency loss L_KL (ablation)ORION (MLP head, 20M warm-up) · MMStaraccuracy: 61.0 -> 63.2Improved
add representation consistency loss L_KL (ablation)ORION (MLP head, 20M warm-up) · GenEvaloverall score: 0.79 -> 0.82Improved
replace linear vision head with MLP head (ablation)ORION without L_KL (20M warm-up) · MMBenchaccuracy: 71.5 -> 79.8Improved
replace linear vision head with MLP head (ablation)ORION without L_KL (20M warm-up) · GenEvaloverall score: 0.65 -> 0.79Improved
MLP head instead of Q-Former at 20M warm-up data (ablation)ORION without L_KL · MMBenchaccuracy: 78.6 -> 79.8Improved
MLP head instead of Q-Former at 5M warm-up data (ablation)ORION without L_KL · MMBenchaccuracy: 77.3 -> 76.4Hurt
scale vision-head warm-up data from 5M to 20M (ablation)ORION MLP head without L_KL · GenEvaloverall score: 0.76 -> 0.79Improved
skip vision-head warm-up (no stage-1 data) (ablation)ORION MLP head without L_KL · MMBenchaccuracy: 76.4 (5M) vs 74.1 (none)Hurt
generative fine-tune of Qwen2.5-VL vs base understandingORION · MMBenchaccuracy: 79.1 -> 83.7 (MMBench), 67.1 -> 64.5 (MM-Vet)Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy