Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Accuracy and Normalized Accuracy under Length Bias: Analysis, Guidelines, and a Bayesian Alternative
Koen Oostermeijer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Bayesian length-corrected scoring instead of byte-normalized accuracy | likelihood-based MC evaluation · 8 benchmarks, Llama 3.1 8B | mean |tau|: 0.27 -> 0.03 | Improved |
| use Bayesian scoring instead of standard accuracy | likelihood-based MC evaluation · 8 benchmarks, Llama 3.1 8B | mean |tau|: 0.20 -> 0.03 | Improved |
| use Bayesian scoring instead of PMI/ANPMI | likelihood-based MC evaluation · 8 benchmarks, Llama 3.1 8B | mean |tau|: 0.17 (PMI), 0.11 (ANPMI) -> 0.03 | Improved |
| byte-normalize log-likelihood instead of raw | standard accuracy · ARC, Llama 3.1 8B | Kendall tau: -0.15 -> 0.22 | Hurt |
| byte-normalize log-likelihood instead of raw | standard accuracy · HellaSwag, Llama 3.1 8B | Kendall tau: -0.49 -> 0.05 | Improved |
| byte-normalize log-likelihood on instruction-tuned models | standard accuracy · 8 benchmarks, Qwen3 instruct | mean |tau|: 0.17 -> 0.38 (Qwen3 8B Instruct) | Hurt |
| increase model size (ablation) | standard accuracy · Llama family | length bias: mean |tau| 0.24 (1B) -> 0.18 (70B) | Improved |
| Bayesian scoring on instruction-tuned small models | Bayesian accuracy · Qwen3 1.7B/4B Instruct | mean |tau|: 0.06 vs ANPMI 0.06 / 0.05 | No clear effect |
Closest papers by meaning
- Evaluating language models as risk scores (NeurIPS 2024 · similarity 0.86)
- Majority of the Bests: Improving Best-of-N via Bootstrapping (NeurIPS 2025 · similarity 0.85)
- B-score: Detecting biases in large language models using response history (ICML 2025 · similarity 0.85)
- Large Language Models Are Not Robust Multiple Choice Selectors (ICLR 2024 · similarity 0.85)
- When Confidence Rises Too Early: Detecting Shortcut Reasoning via Premature Answer Commitment (NeurIPS 2026 · similarity 0.84)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
