Research map / Large language models / Alignment and preferences

Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations

NeurIPS 2025 · Large language models › Alignment and preferences · approach: LLM-as-judge and feedback data

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LAGER, a post-hoc, plug-and-play method that aggregates score-token logits across transformer layers and computes an expected score from a softmax distribution, with optional lightweight tuning of layer weights, to improve LLM-as-a-judge alignment with human scores.

Problem

Point-wise LLM-as-a-judge scores, taken from the final layer's most probable token, correlate weakly with human judgments; improving alignment usually requires complex prompts, reasoning, or fine-tuning.

Approach

Aggregate score-token logits from all decoder layers with learned (or uniform) weights, apply a softmax over candidate scores, and output the expected score; the LLM stays frozen and next-token predictions are unchanged.

Key results

• LAGER (w. tuning) with Mistral-Small-24B and LLaMA-3.3-70B achieves average Spearman 0.540 and 0.570 across six settings, outperforming VScore and E-Score on the same backbones.
• On Flask, HelpSteer and BIGGen Bench, LAGER with tuning improves over the best baseline by up to 7.5% Spearman, per the abstract.
• Without tuning, LAGER outperforms E-Score in 5 of 6 backbones with an average 4.5% gain; on AlpacaEval 2.0, selecting top 10% by LAGER (w. tuning) gives 12.65 LC win rate versus 11.82 for the best baseline (longest responses).

Measured effects reported in the paper

Surprising findings

• Reasoning-first prompting does not consistently help: LAGER with tuning performs weaker under reasoning than under direct scoring, and E-Score is mixed.
• Bottom layers show low or negative correlation with human scores, while middle-upper layers (around 20-30) correlate best, and the final layer drops.

Limitations

• Requires access to intermediate hidden states, so it applies only to open-weight models and not to closed API models.
• Gains are sometimes small or negative relative to E-Score on some backbones and reasoning settings.
• Evaluated only on point-wise scoring without reference responses and on a limited set of benchmarks.

Experimental setup

Datasets: FLASK HelpSteer BIGGen Bench EQ-Bench SelfAware alpaca-cleaned-52k AlpacaEval 2.0

Benchmarks: FLASK HelpSteer BIGGen Bench AlpacaEval 2.0 EQ-Bench SelfAware

Metrics: Spearman correlation Pearson correlation KL divergence MSE LC win rate F1

Baselines: GPTScore Vanilla score (VScore) Expectation score (E-Score) GPT-4o-mini TIGERScore-7B Prometheus2-7B SuperFiltering

Builds on: G-Eval GPTScore Prometheus 2 SuperFiltering

Closest papers by meaning

• Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.90)
• Evaluating the Evaluator: Measuring LLMs’ Adherence to Task Evaluation Instructions (AAAI 2025 · similarity 0.90)
• Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarking (ICLR 2025 · similarity 0.90)
• JudgeBench: A Benchmark for Evaluating LLM-Based Judges (ICLR 2025 · similarity 0.90)
• On Evaluating LLM Alignment by Evaluating LLMs as Judges (NeurIPS 2025 · similarity 0.90)
• Validating LLM-as-a-Judge Systems under Rating Indeterminacy (NeurIPS 2025 · similarity 0.89)