AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Correct Looks Better: Pairwise Comparisons Reveal Accuracy Rankings

Mina Remeli, Moritz Hardt

ICML 2026 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
rank by Bradley-Terry over pairwise judgments instead of direct correctness judginggpt-oss-20b as weak judge · SimpleQAKendall distance to accuracy ranking: 38.2% -> 20%Improved
rank by Bradley-Terry instead of direct judge, weak judgegpt-oss-20b · SimpleQAKendall tau: 0.178 -> 0.667Improved
rank by Bradley-Terry instead of direct judge, strong judgeo3 · SimpleQAKendall tau: 0.922 -> 0.957No clear effect
rank by Bradley-Terry instead of direct judge, averagedjudge o3 · 5 benchmarks averageKendall tau: 0.85 -> 0.81No clear effect
rank by Bradley-Terry instead of direct judge, averagedjudge gpt-oss-20b · 5 benchmarks averageKendall tau: 0.76 -> 0.80Improved
add bias correction (style, self-preference) (ablation)Bradley-Terry ranking · 5 benchmarksKendall tauNo clear effect
restrict to non-discriminative pairs only (ablation)Bradley-Terry ranking · 5 benchmarks averageKendall tau: 0.8 -> 0.68Hurt
restrict to discriminative pairs only (ablation)Bradley-Terry ranking · GSM8KKendall distance: 7.4% -> 3.4%Improved
restrict to discriminative pairs only (ablation)Bradley-Terry ranking · BBHKendall distance: 7.7% -> 3.8%Improved
append echo (repeated Q-A) to answer A (ablation)o3 judge · BBH non-discriminative pairsP(A preferred over B): 0.65 -> 0.29Hurt
append echo to answer B (ablation)o3 judge · BBH non-discriminative pairsP(A preferred over B): 0.65 -> 0.81Improved
append echo on discriminative pairs (ablation)o3 judge · BBH discriminative pairspreference contrast: -0.07No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy