Research map / Large language models / Language, safety and interpretability
RubricRobustness: Evaluating the Sensitivity of Rubrics-Based Benchmarks to Simple Perturbations
Manasi Sharma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| append three targeted checklist items (padding, inversion, missing content) to rubric (ablation) | HealthBench · HealthBench addition perturbation | degradation slope: +147% | Improved |
| append three targeted checklist items to rubric (ablation) | HealthBench · HealthBench negation perturbation | degradation slope: +52% | Improved |
| append three targeted checklist items to rubric (ablation) | WildBench · WildBench addition perturbation | degradation slope: +16% | Improved |
| append three targeted checklist items to rubric (ablation) | WildBench · WildBench negation perturbation | degradation slope: +2% | No clear effect |
| append three targeted checklist items to rubric (ablation) | HealthBench and WildBench · clean responses | benchmark score | No clear effect |
| compare negation with addition perturbation | GPT-5.2 on HealthBench · HealthBench | regression slope: -0.1087 -> -0.3758 | Hurt |
| compare negation with addition perturbation | GPT-5.2 on WildBench · WildBench | regression slope: -0.2795 -> -0.5523 | Hurt |
| compare negation with addition perturbation | Claude Opus 4.5 on WildBench · WildBench | regression slope: -0.3168 -> -0.5795 | Hurt |
| compare deletion with negation perturbation | GPT-5.2 on WildBench · WildBench | 25% drop threshold: 0.57 -> 0.37 | Hurt |
| compare deletion with addition perturbation | GPT-5.2 on HealthBench · HealthBench | 25% drop threshold: 0.38 -> 0.32 | Hurt |
| perturb 25% of sentences by deletion | WildBench responses · WildBench | fraction of cases with score increase: 7.0% | Mixed |
Closest papers by meaning
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.88)
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.88)
- CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning (NeurIPS 2025 · similarity 0.88)
- Any Large Language Model Can Be a Reliable Judge: Debiasing with a Reasoning-based Bias Detector (NeurIPS 2025 · similarity 0.88)
- Brittlebench: Quantifying LLM robustness via prompt sensitivity (NeurIPS 2026 · similarity 0.88)
- Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
