AI Paper Atlas

Research map / Large language models / Alignment and preferences

Predicting Human-Gain Curves under Local Proxy Optimization from Repeated Ratings

Moonwon Choi, Kisung Nam, Seunggeun Lee

NeurIPS 2026 · Large language models › Alignment and preferences · approach: Robust and calibrated DPO · 0 citations (OpenAlex)

Read the paper

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a pre-optimization diagnostic that uses repeated human ratings to predict how human-perceived quality changes as a fixed proxy objective is optimized more strongly.

Problem

Average proxy-human correlation does not reveal how human-perceived quality changes as optimization strength increases under a proxy objective.

Approach

Models proxy optimization locally and uses repeated human ratings to predict the human-gain curve induced by a fixed proxy before optimization is performed.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add curvature term B to A-only forecast (ablation)PACE · SummEval humanendpoint MAE at d=0.10: 0.0310 -> 0.0174Improved
add curvature term B to A-only forecast (ablation)PACE · WMT MQMendpoint MAE at d=0.10: 0.1298 -> 0.0846Improved
add curvature term B to A-only forecast (ablation)PACE · OpenMEVA-MAGSpre-pool CurveMAE: 0.01058 -> 0.00991No clear effect
add curvature term B to A-only forecast (ablation)PACE · SummEval, GPT-4o, WMT, OpenMEVACurveMAE reduction: 34.3%, 21.7%, 26.4%, 6.4%Improved
use proxy-human alignment instead of reliability-only A_nu (ablation)PACE · WMT MQMendpoint MAE at d=0.10: 0.6230 -> 0.0846Improved
replace PACE with binned calibration (ablation)Flexible calibration · SummEval humanpre-pool CurveMAE: 0.01185 -> 0.03010Hurt
replace PACE with isotonic regression (ablation)Flexible calibration · WMT MQMpre-pool CurveMAE: 0.05921 -> 0.05882No clear effect
provide target-pool proxy scores to isotonic calibratorIsotonic calibration · SummEval humantarget-pool-aware CurveMAE: 0.01185 -> 0.00909Improved
use projection instead of direct integration (ablation)Calibrated curve · 12 dataset-calibrator cellsCurveMAE: 0.00002-0.00286 differenceNo clear effect
add target calibration to source-only coefficientsPACE transfer · SummEval to WMTendpoint MAE: 0.0436 -> 0.0259Improved
add second expert rating to calibrationPACE B estimate · SummEvalmean absolute error of B-hat: 0.0618 -> 0.0428Improved
1% positive 8-SD proxy contaminationPACE B estimate · OpenMEVAcorrelation with clean B: -0.11Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy