Research map / Large language models / Retrieval and knowledge
LLMEval: A Preliminary Study on How to Evaluate Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compares how to evaluate LLMs (criteria, onsite/crowd/public/GPT-4 annotators, star vs pairwise scoring, Elo vs points ranking) and releases LLMEval-1 and LLMEval-2 with 243k manual and 57k GPT-4 annotations over 20 LLMs.
Problem
Existing LLM evaluation research focuses on what tasks and which knowledge domains to test, but not on how to evaluate: which criteria, annotator types, scoring methods, and ranking systems to use.
Approach
Systematically compares five evaluation criteria, three human annotator types plus GPT-4, star scoring versus pairwise comparison, and Elo versus points ranking, and analyzes their accuracy, consistency, alignment, and stability to derive practical conclusions.
Key results
- Onsite star scoring achieved average accuracy 0.892 versus GPT-4 star scoring 0.908; GPT-4 pairwise accuracy only 0.688.
- Star-scoring rank alignment between GPT-4 and manual evaluation reached Spearman rho 0.949, higher than the manual star vs pairwise correlation of 0.938.
- GPT-4 chose the longer response 78.8% of the time when length difference was at least 300 characters, versus 51.4% for humans.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use onsite annotators instead of crowd-sourcing or public annotators | Manual evaluation · LLMEval-1 | accuracy and consistency: 0.892 average accuracy | Helped |
| Use GPT-4 star scoring instead of GPT-4 pairwise comparison | GPT-4 evaluation · LLMEval-1 | accuracy: 0.688 -> 0.908 | Helped |
| Use star scoring instead of pairwise comparison | GPT-4 and manual evaluation · LLMEval-1 | Spearman's correlation coefficient (ρ) with manual evaluation: 0.902 -> 0.949 | Helped |
| Use GPT-4 rather than human annotators for pairwise comparisons | Pairwise response evaluation · LLMEval-1, Δ length ≥ 300 | longer response selected as win: 51.4% -> 78.8% | Mixed |
| Evaluate subjective rather than objective questions | Manual vs. GPT-4 evaluation · LLMEval-2 | proportion of questions with accuracy score difference ≥ 2: 12.98% -> 37.05% | Hurt |
| Provide answer hints to annotators (ablation) | Manual annotation · LLMEval-1 | average score: 9.79% higher without hints | Hurt |
| Use Elo ratings rather than Points scoring | Pairwise-comparison ranking · LLMEval-1 | ranking stability across match order: 1 to 3 | Hurt |
| Use Elo ratings for 100,000 rounds of pairwise comparisons | Pairwise-comparison ranking · LLMEval-1 | ranking stability: 100,000 rounds | Hurt |
| Evaluate informativeness and accuracy as criteria | LLM evaluation · LLMEval-1 | between-model score separation: 0.853; 0.776 | Helped |
Surprising findings
- Elo rating ranks keep fluctuating even after 100,000 comparison rounds and are sensitive to match order, with GPT-4 rank shifting between 1 and 3 under shuffling.
- GPT-4 star-scoring rankings align with human rankings more closely than human star scoring aligns with human pairwise comparison.
Limitations
- Elo variance analysis relies on a simplified linear approximation and assumes i.i.d. outcomes.
- Evaluation is in Chinese and uses a limited set of models and tasks (453 questions in LLMEval-1, 480 in LLMEval-2).
Experimental setup
Datasets: LLMEval-1 LLMEval-2
Benchmarks: LLMEval-1 LLMEval-2
Metrics: accuracy vs ground truth annotator consistency Spearman rho Kendall tau length bias rate Elo rating variance
Baselines: HELM MMLU C-Eval AGIEval BERTScore AlpacaFarm Chatbot Arena
Builds on: LLM-as-a-Judge AlpacaFarm Chatbot Arena
Closest papers by meaning
- tinyBenchmarks: evaluating LLMs with fewer examples (ICML 2024 · similarity 0.93)
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.92)
- Neither Valid nor Reliable? Investigating the Use of LLMs as Judges (NeurIPS 2025 · similarity 0.91)
- BenchmarkCards: Standardized Documentation for Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.91)
- Fixing It in Post: A Comparative Study of LLM Post-Training Data Quality and Model Performance (NeurIPS 2025 · similarity 0.91)
- RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.