Research map / Large language models / Instruction and fine-tuning
JudgeLM: Fine-tuned Large Language Models are Scalable Judges
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes Vicuna-based 7B/13B/33B LLMs as open-source judges (JudgeLM) on a 100K GPT-4-annotated open-ended judging dataset, using swap augmentation, reference support, and reference drop to reduce position, knowledge, and format biases.
Problem
Evaluating LLMs in open-ended scenarios is hard: existing benchmarks and traditional metrics are inadequate, GPT-4 judges raise cost and reproducibility concerns, and prior fine-tuned open judges (PandaLM) suffer from data quality and LLM biases.
Approach
Fine-tune open Vicuna models as pairwise judges on a large GPT-4-labeled dataset, augmenting training with answer-swapped samples (swap augmentation), with and without reference answers (reference support and reference drop), and use a score-first output format so that judgments are produced quickly without generating reasons.
Key results
- JudgeLM-33B reaches 89.03% agreement with GPT-4 and 91.36% consistency under swapping on the JudgeLM val set, versus 73.83% for GPT-3.5 and 74.86% for Auto-J-13B.
- JudgeLM-7B judges 5K pairs in about 3 minutes on 8 A100 GPUs when reasons are not generated.
- On the PandaLM test set, JudgeLM-33B reaches 75.18% agreement with human labels, above GPT-4's reported 66.47%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale JudgeLM from 7B to 33B | JudgeLM · JudgeLM val, without reference | agreement with GPT-4: 81.11 -> 89.03 | Helped |
| Increase training data from 3.5k to 100k samples | JudgeLM-7B · JudgeLM val | agreement with GPT-4: 75.87 -> 83.71 | Helped |
| Add swap augmentation (ablation) | JudgeLM-7B · JudgeLM val, without reference | consistency with swapped answers: 73.45 -> 78.89 | Helped |
| Add swap augmentation (ablation) | JudgeLM-7B · JudgeLM val, without reference | agreement with GPT-4: 75.87 -> 76.51 | Helped |
| Add reference support and evaluate with matching reference format (ablation) | JudgeLM-7B · JudgeLM val, with reference | agreement with GPT-4: 75.87 -> 80.15 | Helped |
| Use reference drop during fine-tuning (ablation) | JudgeLM-7B · JudgeLM val, mismatched reference formats | agreement with GPT-4: 73.09 -> 80.35 | Helped |
| Use explanation-first CoT instead of score-first (ablation) | JudgeLM-7B · JudgeLM val | agreement with GPT-4: 75.87 -> 75.54 | No clear effect |
| Use explanation-first CoT instead of score-first (ablation) | JudgeLM-7B · JudgeLM val | consistency with swapped answers: 73.45 -> 74.39 | Helped |
| Use JudgeLM-33B instead of GPT-4 (0-shot) | multimodal answer judging · MM-Vet human-annotated benchmark | agreement with human: 86.70 -> 91.74 | Helped |
| Use JudgeLM-33B instead of OpenAI Moderation | ToxicChat classification · ToxicChat test set (0124) | precision: 54.76 -> 58.79 | Helped |
| Use JudgeLM-7B instead of GPT-3.5-turbo-0125 | reward-model evaluation · RewardBench | Score: 64.5 -> 78.5 | Helped |
Surprising findings
- Fine-tuned JudgeLM-33B surpasses its GPT-4 teacher on PandaLM test-set agreement with human labels.
- Judges fine-tuned without references perform poorly when references are provided, and vice versa (format bias).
Limitations
- Dataset construction cost about $4000 in GPT-4 API calls limits scaling of the training data.
- Only answers from open-source LLMs are included; closed-source model answers are left to future work.
- Position bias remains (bias toward 1st), and math judging is somewhat weaker than coding and common-sense.
Experimental setup
Datasets: Alpaca-GPT4 Dolly-15K GPT4All-LAION ShareGPT PandaLM train/test sets
Benchmarks: JudgeLM val set PandaLM test set MM-Vet (human-annotated) ToxicChat RewardBench
Metrics: agreement precision recall F1 consistency bias toward 1st bias toward 2nd delta bias
Baselines: GPT-3.5 GPT-4 PandaLM-7B Auto-J-13B InstructScore-7B Vicuna-13B
Builds on: Vicuna PandaLM MT-bench Chain-of-thought
Closest papers by meaning
- Tuning LLM Judge Design Decisions for 1/1000 of the Cost (ICML 2025 · similarity 0.92)
- MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark (ICML 2024 · similarity 0.92)
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.92)
- MixEval: Deriving Wisdom of the Crowd from LLM Benchmark Mixtures (NeurIPS 2024 · similarity 0.91)
- tinyBenchmarks: evaluating LLMs with fewer examples (ICML 2024 · similarity 0.91)
- RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.