AI Paper Atlas

Research map / Large language models / AI and society

AtC: Aggregate-then-Calibrate for Human-centered Assessment

Zejun Xie, Xintong Li, Guang Wang, Desheng Zhang

ICLR 2026 · Large language models › AI and society · approach: Data commons and human baselines

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
calibrate model scores onto the human consensus ranking (AtC) vs model-onlyAtC with HRA-E · Reading-level (semi-synthetic)MSE: 29.00 -> 8.191Improved
calibrate model scores onto the consensus ranking vs model-onlyAtC with HRA-G · Reading-level (semi-synthetic)Wasserstein: 2.831 -> 0.839Improved
calibrate model scores onto consensus ranking vs human-onlyAtC with HRA-E · Reading-level (semi-synthetic)Kendall tau: 0.375 -> 0.403Improved
use heterogeneous Stage-1 (HRA-G) instead of homogeneous BTL (ablation)AtC · Reading-level (semi-synthetic)Kendall tau: 0.373 -> 0.410Improved
use heterogeneous Stage-1 (HRA-G) instead of homogeneous TCV (ablation)AtC · Reading-level (semi-synthetic)Wasserstein: 0.914 -> 0.839Improved
heterogeneous HRA human-only scores versus calibrated, on distance metrichuman-only s* · Reading-level (semi-synthetic)Wasserstein: 2.250 vs 0.839Hurt
calibrate onto consensus ranking vs human-onlyAtC with HRA-E · Dots-activityKendall tau: 0.922 -> 0.943Improved
calibrate onto consensus ranking vs model-onlyAtC with HRA-E · Dots-activityMSE: 11.75 -> 9.59Improved
AtC vs GPPLAtC with HRA-E · Dots-activityMSE: 4220.36 -> 9.59Improved
add 400 or fewer pairwise inversions to consensus (ablation)AtC · Reading-level (semi-synthetic)Kendall tau: decreases steadilyHurt
add more than 500 pairwise inversions to consensus (ablation)AtC · Reading-level (semi-synthetic)Kendall tau: becomes negativeHurt
calibrate with aggregated ratings instead of rankings (ablation)AtC · Dots-activityKendall tauHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy