AI Paper Atlas

Research map / Large language models / Alignment and preferences

SoliReward: Mitigating Susceptibility to Reward Hacking and Annotation Noise in Video Generation Reward Models

Jiesong Lian, Ruizhe Zhong, Zixiang Zhou, Xiaoyue Mi, Long Hu, Yuan Zhou, Qinglin Lu, Yixue Hao, Junchi Yan

CVPR 2026 · Large language models › Alignment and preferences · approach: Reward model design and preference models · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add win-tie pairs (BT-WT loss) to Bradley-Terry training (ablation)SoliReward RM · HunyuanVideo post-training, VBench2 Human FidelityVBench2: 0.8693 -> 0.8999Improved
add win-tie pairs (BT-WT loss) (ablation)SoliReward RM · HunyuanVideo post-training, VideoAlign MQMQ: 0.1719 -> 0.3302Improved
add win-tie pairs (BT-WT loss) (ablation)SoliReward RM · Phy & Deform RM accuracyRM accuracy (%): 77.63 -> 78.27No clear effect
replace Linear/token-logit/special-token head with HPQA multi-layer query attention (ablation)SoliReward RM · Phy & Deform OODRM accuracy (%): 78.66 -> 80.08 (best alternative)Improved
use HPQA instead of linear head (ablation)SoliReward RM · TA (semantic alignment) OODRM accuracy (%): 31.92 -> 60.25Improved
use HPQA instead of special token + linear (ablation)SoliReward RM · Phy & Deform OODRM accuracy (%): 73.61 -> 80.08Improved
add BCE penalty to BT-WT loss (ablation)SoliReward RM · Phy & Deformreward margin: 3.72 -> 2.97Hurt
add BCE penalty to BT-WT loss (ablation)SoliReward RM · HunyuanVideo post-trainingMQ: 0.3302 -> 0.1154Hurt
use single-item binary annotation instead of pairwise comparison (ablation)annotation pipeline · Inter-annotator agreementKrippendorff alpha: 0.3516 -> 0.4939Improved
use cross-prompt instead of in-prompt pairing (ablation)SoliReward RM · RM accuracyRM accuracyNo clear effect
guide DanceGRPO with SoliReward instead of VideoAlign MQHunyuanVideo · DanceGRPO post-trainingVBench2 Human Fidelity: 0.8695 -> 0.8999Improved
SoliReward vs VideoPhy on semantic alignment OODreward model · TA OODRM accuracy (%): 60.52 -> 60.25No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy