Research map / Large language models / Language, safety and interpretability
EvalAssist: LLM-as-a-Judge Simplified
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Presents EvalAssist, an online criteria-development environment and Python toolkit for LLM-as-a-judge evaluation with token-probability judgement, positional bias checking, and certainty estimation, deployed internally to 700+ users.
Problem
LLM-as-a-judge evaluation is hard to build, test, and apply reliably; issues include bias, prompt sensitivity, hallucinated or malformed judgements, and low trust in automated evaluators.
Approach
A combined criteria authoring/testing UI and portable JSON criteria format plus a library of evaluators that pick judgements via token log-probabilities over predefined options, with positional-shuffle consistency checks and certainty estimates.
Key results
- No single evaluator outperforms on all tasks; LLM evaluators do better on simpler criteria and on pairwise than direct assessment.
- EvalAssist is deployed internally with over 700 users.
- User research finds direct assessment preferred for control and pairwise preferred for subjective tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use pairwise assessment instead of direct assessment | EvalAssist evaluators · Benchmarks with human annotations | evaluator performance | Helped |
| Evaluate with simpler criteria rather than criteria requiring nuanced understanding | LLM evaluators · Benchmarks with human annotations | evaluator performance | Helped |
| Use token log-probability option selection instead of direct generation | EvalAssist evaluators · LLM-as-a-judge evaluations | robustness | Helped |
| Use different evaluator models across tasks rather than assuming one evaluator is best | EvalAssist evaluators · Several datasets and benchmarks | evaluator performance | Mixed |
Surprising findings
- Users tended to over-fit evaluation criteria when working with a single task context and a small set of responses.
- For complex criteria, LLM evaluators tend to be pessimistic relative to humans, and human-human agreement is also low in those cases.
Limitations
- Token-probability judgement is more expensive than direct generation.
- No single evaluator dominates across tasks, requiring users to experiment.
- Users over-fit criteria to small sets of task contexts and responses.
Experimental setup
Datasets: human-annotated evaluation benchmarks (Bavaresco et al. 2024)
Benchmarks: benchmarks from Bavaresco et al. 2024 with human annotations
Metrics: agreement with human annotations
Builds on: Chain-of-Thought prompting Unitxt Prometheus 2
Closest papers by meaning
- JudgeBench: A Benchmark for Evaluating LLM-Based Judges (ICLR 2025 · similarity 0.88)
- Evaluating the Evaluator: Measuring LLMs’ Adherence to Task Evaluation Instructions (AAAI 2025 · similarity 0.88)
- Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators (ICML 2025 · similarity 0.87)
- Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judge (ICML 2025 · similarity 0.86)
- Validating LLM-as-a-Judge Systems under Rating Indeterminacy (NeurIPS 2025 · similarity 0.85)
- Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.