Research map / Large language models / Language, safety and interpretability

Benchmarking LLMs' Judgments with No Gold Standard

ICLR 2025 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GEM, a generative mutual-information (PMI-based) metric that scores LLM outputs against non-gold peer references, plus GRE-bench for benchmarking LLM peer-review generation on ICLR 2023 data.

Problem

Evaluating open-ended, subjective LLM generations (e.g., peer reviews) when no gold-standard reference exists, while remaining accurate and resistant to superficial manipulations such as rephrasing or elongation.

Approach

Estimate the pointwise mutual information between a candidate response and a peer reference response using an LLM's token probabilities (log Pr(y|x) minus log Pr(y)), after preprocessing both texts to strip style; GEM-S conditions on a task synopsis (e.g., abstract) to count only additional semantic information.</µcore_idea>

Key results

• GEM-S achieves Spearman rho 0.479 with instructor grades vs 0.537 for GPT-4o LMExaminer; all other baselines except BARTScore-recall fail significant correlation.
• GEM and GEM-S show significant negative SMD on all three degradations (e.g., GEM-S -0.566 on abstract-only review), while LMExaminer fails to penalize abstract-only reviews (0.715).
• GEM and GEM-S show no significant score increase after GPT-4o rephrase, Llama-3.1 rephrase, or meaningless elongation, whereas LMExaminer increases significantly after all manipulations.

Measured effects reported in the paper

Surprising findings

• LMExaminer (GPT-4o) failed to penalize abstract-only fictitious reviews (SMD +0.715), while GEM-S penalized them strongly.
• GRE-bench scores with an 8B evaluator correlate highly with a 70B evaluator (Spearman 0.93 for GEM), suggesting small evaluation LMs suffice.

Limitations

• Theoretical guarantees rely on the LLM estimating the true conditional distribution accurately, which may not hold.
• Requires reference responses containing original human-generated information; LLM-only references can unfairly favor the generating model.
• Preprocessing model choice materially affects results (Llama-3.2 90B preprocessing lowers human correlation from 0.433 to 0.274 and weakens manipulation robustness).

Experimental setup

Datasets: ICLR 2023 peer reviews (OpenReview) Human-annotated proposal peer grading dataset (30 proposals, ~180 reviews) Yelp restaurant reviews

Benchmarks: GRE-bench (ICLR 2023) Human-annotated peer grading dataset Yelp restaurant reviews

Metrics: Spearman correlation standardized mean difference (Cohen's d) GRE-bench score

Baselines: BLEU ROUGE-L BERTScore BARTScore LMExaminer (GPT-4o) GEM-raw

Builds on: BARTScore GPTScore peer prediction information elicitation without verification LM-as-examiner

Closest papers by meaning

• Neither Valid nor Reliable? Investigating the Use of LLMs as Judges (NeurIPS 2025 · similarity 0.91)
• Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models (ICLR 2024 · similarity 0.90)
• Accelerating Unbiased LLM Evaluation via Synthetic Feedback (ICML 2025 · similarity 0.90)
• RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.90)
• PiCO: Peer Review in LLMs based on Consistency Optimization (ICLR 2025 · similarity 0.90)
• RevisEval: Improving LLM-as-a-Judge via Response-Adapted References (ICLR 2025 · similarity 0.89)