AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

GenAlign: Towards Unified Alignment Framework of MLLMs via Generative Reward Model

Jingyu Zhang, Kun Yang, Ming Wen, Jiawei Zhao, Yuxuan Liu, Zhuoer Xu, shiwen cui

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GenAlign, a unified framework for aligning MLLMs with human preferences that builds on a generative reward model to address position bias and computational overhead.

Problem

Aligning multimodal large language models with human preferences; existing generative reward models suffer from severe position bias and high computational overhead.

Approach

Combines a robust generative reward model (designed to reduce position bias) with an alignment procedure in a unified framework, aiming to make reasoning-based reward modeling efficient for MLLMs. Specific mechanism is not stated in the abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Online Position Debiasing (OPD) to GRM training (ablation)GenAlign GRM · reward benchmarksposition gap / order-invariant accuracyImproved
add bidirectional comparison to reward estimation (ablation)GenAlign alignment · HaloQuest, LLaVA-1.5-7Baccuracy: 30.45 -> 45.16Improved
add bidirectional comparison (ablation)GenAlign alignment · WildVision, Qwen2.5-VL-7Bwin rate: 42.90 -> 45.00Improved
add advantage smoothing (ablation)GenAlign alignment · WildVision, Qwen2.5-VL-7Bwin rate: 43.80 -> 45.00Improved
add advantage smoothing (ablation)GenAlign alignment · HaloQuest, LLaVA-1.5-7Baccuracy: 39.20 -> 45.16Improved
advantage smoothing instead of hard clipping (ablation)GenAlign on Qwen2.5-VL-7B · HaloQuestaccuracy: 77.93 -> 81.99Improved
advantage smoothing instead of hard clipping (ablation)GenAlign on Qwen2.5-VL-7B · MathVistaaccuracy: 67.2 vs 67.4Mixed
increase number of reference responses K from 1 to 16 (all-to-all) (ablation)GenAlign dynamic reference anchoring · MLLM benchmarksperformanceNo clear effect
dynamic single-reference anchoring instead of exhaustive all-to-all comparisons (ablation)GRM-based alignment · per-batch latencylatency (s): 320s -> 101sImproved
increase inference samples N with majority voting / any-correct (ablation)GenAlign GRM · VL-Reward and MM-RLHF benchmarksaccuracyImproved
align with GenAlign reward instead of scalar RMQwen3-VL-8B · MathVistaaccuracy: 72.6 -> 73.4Mixed
align with GenAlign rewardLlava-1.5-7B · HaloQuestaccuracy: 14.52 -> 45.16Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy