AI Paper Atlas

Research map / Large language models / Alignment and preferences

Reward Auditor: Inference on Reward Modeling Suitability in Real-World Perturbed Scenarios

Jianxiang Zang, Yongda Wei, Ruxue Bai, Shiyu Jiang, Nijia Mo, Binhong Li, Qiang Sun, Hui Liu

ICML 2026 · Large language models › Alignment and preferences · approach: Reward model design and preference models · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply synonym transformation to responsesmost reward models · RM-Bench Chatpreference confidence: e.g. URM-LLaMa-3-8B 0.370 effect sizeHurt
translate responses to another languagemost reward models · RM-Bench Chatpreference confidence: e.g. URM-LLaMa-3-8B 0.490, Llama-3-OffsetBias-RM-8B 0.741Hurt
restructure/extend responses (structured presentation, length extension)most reward models · RM-Bench Chatpreference confidence: SOLAR 0.774/0.906 vs Skywork-Reward-V2-Llama-3.1-8B -0.382/-0.489Mixed
apply controlled prompt perturbations (typos, formatting, irrelevant tokens)reward models · RM-Bench Chatpreference confidenceMixed
move evaluation from objective to subjective domainsreward models · RM-Bench subsetssuitability degradationHurt
use low-suitability-risk RM for PPO instead of high-risk RMLlama3-8B policy · perturbed vs clean RLHFperturbed policy win rate: Spearman -0.881Improved
add an irrelevant username or weblink to promptFsfairX-LLaMA3-RM-v0.1 · RM-Bench Chatpreference confidence: IU 0.502Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy