Research map / Large language models / Instruction and fine-tuning
Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RBD, a plug-in reasoning-based bias detector fine-tuned from distilled reasoning traces to flag biased LLM-judge evaluations and drive evaluator self-correction across four bias types.
Problem
LLM-as-a-Judge evaluations are susceptible to biases (verbosity, position, bandwagon, sentiment); existing prompting fixes are weak for deep-rooted biases and fine-tuning judges is impossible for closed-source evaluators.
Approach
Instead of modifying the evaluator, train an external reasoning model (RBD) on distilled teacher reasoning to detect biased judgments and generate structured bias analysis that is fed back to the evaluator for iterative revision.
Key results
- RBD-8B raises average accuracy by 18.5% and consistency by 10.9% across 8 evaluators and 4 biases (abstract); surpasses prompting baselines by 12.8%.
- RBD-14B reaches 0.828 accuracy on the RBD test set, above zero-shot DeepSeek-R1 (0.762).
- On LLaMA-3.1-70B, RBD-8B gives verbosity accuracy 0.940 vs 0.562 vanilla, outperforming Prometheus2 and Skywork baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RBD-8B to LLM evaluators | LLM evaluators · four bias types | accuracy and consistency: verbosity accuracy +37.2%, consistency +26.2%; position consistency +7.2%; bandwagon accuracy +9.9%, consistency +4.1%; | Helped |
| Increase RBD size from 1.5B to 14B | RBD with LLM evaluators · four bias types | accuracy gain: +6.5% to +15.4% | Helped |
| Increase RBD size from 1.5B to 14B | RBD with LLM evaluators · four bias types | consistency gain: +5.3% to +11.9% | Helped |
| Use RBD-8B instead of no RBD | LLM evaluators on FactQA · FactQA, verbosity bias | accuracy: 0.712 to 0.796 | Helped |
| Use RBD-14B instead of no RBD | LLM evaluators on FactQA · FactQA, verbosity bias | accuracy: 0.712 to 0.882 | Helped |
| Provide full reasoning trace instead of only bias label (ablation) | RBD-8B with GPT-4o evaluator · four bias types | accuracy (position uses consistency): 0.716→0.912; 0.544→0.610; 0.518→0.590; 0.824→0.852 | Helped |
| Remove evaluator model name and specific bias type from the prompt (ablation) | RBD · prompt robustness evaluation | performance | No clear effect |
| Train with bias labels only instead of reasoning supervision (ablation) | RBD training · reconstructed verbosity diagnostic set | accuracy: zero accuracy | Hurt |
| Add RBD-8B | LLM evaluators on LLMBar · verbosity bias | average accuracy: 0.681→0.691 | Helped |
| Add RBD-14B | LLM evaluators on LLMBar · verbosity bias | average accuracy: 0.681→0.792 | Helped |
| Add RBD-8B | LLM evaluators on JudgeBench · bandwagon bias | average accuracy: 0.578→0.595 | Helped |
| Add RBD-14B | LLM evaluators on JudgeBench · bandwagon bias | average accuracy: 0.578→0.624 | Helped |
Surprising findings
- Bias-only fine-tuning reaches high accuracy on the original test set but fails catastrophically on diagnostic sets where format cues are reversed.
- Closed-source evaluators are generally more stubborn toward revision than open-source ones, and smaller closed models show the highest overconfidence.
Limitations
- Bias types limited to four structural biases; evaluated on math-centric training data.
- Gains are uneven: position bias accuracy is unchanged, and some RBD sizes slightly hurt sentiment accuracy.
- Stubbornness: many evaluators refuse to revise even after a bias flag.
Experimental setup
Datasets: GSM8K Arena LLMBar JudgeBench FactQA
Benchmarks: bias-specific evaluation sets (verbosity, position, bandwagon, sentiment) LLMBar JudgeBench FactQA
Metrics: accuracy consistency precision recall F1-score
Baselines: zero-shot DeepSeek-R1 in-context debiasing prompt Prometheus2-8x7B Skywork-LLaMA-3.1-70B bias-only fine-tuning
Builds on: DeepSeek-R1-Distill models LLM-as-a-Judge Prometheus
Closest papers by meaning
- Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (ICLR 2025 · similarity 0.91)
- Limits to scalable evaluation at the frontier: LLM as judge won’t beat twice the data (ICLR 2025 · similarity 0.89)
- Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators (ICML 2025 · similarity 0.89)
- Unbiased Evaluation of Large Language Models from a Causal Perspective (ICML 2025 · similarity 0.89)
- Self-correcting for Debiasing Large Language Models (ICML 2026 · similarity 0.89)
- RocketEval: Efficient automated LLM evaluation via grading checklist (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.