Research map / Large language models / Language, safety and interpretability
A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Comprehensively benchmarks 30 LLMs as multi-dimensional dialogue evaluators across 12 meta-evaluation datasets, probes robustness with adversarial perturbations, and tests dimension-wise and model-wise ensembles.
Problem
Existing studies of LLMs as automatic dialogue evaluators are narrow in meta-evaluation datasets, evaluation modes, and LLM coverage, so their effectiveness, especially for open-source models and robustness, remains unclear.
Approach
Systematically correlates 30 LLMs (proprietary and open-source) with human and GPT-4-completed annotations across turn- and dialogue-level dimensions, uses explicit scoring for proprietary models and Yes-probability implicit scoring for open-source ones, and introduces adversarial perturbation robustness tests plus dimension-wise and model-wise ensembling.
Key results
- ChatGPT ranks first on average among 30 LLMs (turn-level avg 0.554, dialogue-level avg 0.644), with Palm-2 Bison second; GPT-4 reaches 0.564 turn-level and 0.669 dialogue-level average on human-annotated data.
- Among open-source models, Tulu-13B is top at the turn level (avg 0.444) and Chimera-13B at the dialogue level, with ChatGPT outperforming Tulu-13B by about 11% on average at turn level.
- Dimension-wise ensembling improves overall-quality correlation for Chimera-13B and Baize-v2-13B at dialogue level by more than 10%, and model-wise ensemble of top-3 open models matches ChatGPT at dialogue level.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Instruction-tune the LLaMA-7B backbone with Alpaca | LLaMA-7B · Turn-level datasets (average) | Pearson correlation: 0.075 -> 0.326 | Helped |
| Instruction-tune the LLaMA-7B backbone with Alpaca | LLaMA-7B · Dialogue-level datasets (average) | Pearson correlation: 0.161 -> 0.525 | Helped |
| Instruction-tune the LLaMA-13B backbone with Tulu | LLaMA-13B · Turn-level datasets (average) | Pearson correlation: 0.353 -> 0.444 | Helped |
| Instruction-tune the LLaMA-13B backbone with Tulu | LLaMA-13B · Dialogue-level datasets (average) | Pearson correlation: 0.382 -> 0.564 | Helped |
| Use more diverse instruction data in Tulu rather than Vicuna | LLaMA-13B instruction-tuned evaluators · Turn-level datasets (average) | Pearson correlation: 0.148 | Helped |
| Use more diverse instruction data in Tulu rather than Vicuna | LLaMA-13B instruction-tuned evaluators · Dialogue-level datasets (average) | Pearson correlation: 0.149 | Helped |
| Average dimension-specific scores instead of direct overall-quality prompting | Chimera-13B · Turn-level overall response quality | Pearson correlation: 0.404 -> 0.476 | Helped |
| Average dimension-specific scores instead of direct overall-quality prompting | Chimera-13B · Dialogue-level overall quality | Pearson correlation: 0.563 -> 0.680 | Helped |
| Average dimension-specific scores instead of direct overall-quality prompting | Baize-v2-13B · Dialogue-level overall quality | Pearson correlation: 0.469 -> 0.578 | Helped |
| Average scores from the top 3 open-source models per dimension | Open-source dialogue evaluators · Dialogue-level datasets (average) | Pearson correlation: 0.605 -> 0.643 | Helped |
| Average scores from the top 3 open-source models per dimension | Open-source dialogue evaluators · Turn-level datasets (average) | Pearson correlation: 0.554 -> 0.496 | Hurt |
| Use the dialogue-level model-wise ensemble instead of Palm-2 Bison | Dialogue-level evaluators · Dialogue-level datasets (average) | Pearson correlation: 0.605 -> 0.643 | Helped |
Surprising findings
- No LLM is robust against all adversarial perturbations; even ChatGPT performs poorly on many perturbations despite strong correlations.
- Baize-v2-13B and WizardLM-13B frequently fail to detect random response perturbations, with robustness ratios near zero for several perturbations.
Limitations
- GPT-4 annotations used to fill missing dimensions may contain biases, and GPT-4 itself does not exceed 0.8 correlation on average.
- Proprietary models' output probabilities are unavailable, so they use explicit scoring while open-source models use implicit Yes-probability scoring, making comparisons inconsistent.
- Robustness analysis limited to seven strong models and perturbations of fixed threshold theta.
Experimental setup
Datasets: Persona-USR Topical-USR Persona-Zhao DailyDialog-Zhao FED-Turn ConTurE-Turn FED-Dial IEval-Dial Persona-See Reliable-Dial ConTurE-Dial Human-Eval
Benchmarks: Persona-USR Topical-USR Persona-Zhao DailyDialog-Zhao FED-Turn ConTurE-Turn FED-Dial IEval-Dial Persona-See Reliable-Dial ConTurE-Dial Human-Eval
Metrics: Pearson correlation Spearman correlation robustness ratio
Baselines: GPT-4 ChatGPT Palm-2 Bison Tulu-13B Chimera-13B WizardLM-13B-V1.2 Baize-v2-13B LLaMA-2-Chat-13B
Builds on: G-Eval FED USR ConTurE
Closest papers by meaning
- SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words (NeurIPS 2024 · similarity 0.89)
- Evaluating Large Language Models at Evaluating Instruction Following (ICLR 2024 · similarity 0.89)
- A Statistical Framework for Ranking LLM-based Chatbots (ICLR 2025 · similarity 0.88)
- Neither Valid nor Reliable? Investigating the Use of LLMs as Judges (NeurIPS 2025 · similarity 0.88)
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.88)
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.