Research map / Large language models / Retrieval and knowledge
PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PertEval, a toolkit that applies knowledge-invariant perturbations (LLM-based paraphrasing and format changes) to MMLU subsets and uses consistency metrics (ACC@Consist, PDR, ROP) to reveal inflated LLM knowledge scores.
Problem
Static close-ended benchmarks overestimate LLM knowledge capacity due to limited test formats and data contamination, so they do not reflect real knowledge mastery.
Approach
Generate on-the-fly perturbed test sets from existing benchmarks that preserve knowledge content (sentence-by-sentence LLM paraphrasing plus format-level changes such as option permutation, option ID shifting, multi-judgment conversion, and question/option position swapping), then measure real knowledge via consistency across raw and perturbed versions (ACC@Consist, PDR, ROP).
Key results
- gpt-4-turbo shows an absolute 25.8% overestimation on MMLU subsets under PertEval; its ACC@Consist is only 0.555.
- Response pattern analysis: gpt-4-turbo's rate of selecting extra incorrect options rises from 12% to 21.7% on perturbed data.
- SwapPos is the most damaging perturbation on average (AVG PDR -0.1637), and all models show negative macro-PDR under knowledge-invariant paraphrasing (KnInvPara).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate static benchmark questions with PertEval composite knowledge-invariant perturbations and consistency | gpt-4-turbo · MMLU subset | ACC@Consist: ACC@Consist of only 0.555 | Hurt |
| Apply knowledge-invariant paraphrasing | gpt-4-turbo · MMLU subset, mastered questions | PDR: AVG_macro -0.0067 | No clear effect |
| Swap question and options positions | gpt-4-turbo · MMLU subset, mastered questions | PDR: AVG_macro -0.0568 | Hurt |
| Apply knowledge-invariant paraphrasing | LLMs evaluated in the paper · MMLU subset | PDR: -0.0411 | Hurt |
| Swap question and options positions | LLMs evaluated in the paper · MMLU subset | PDR: -0.1637 | Hurt |
| Apply option permutation | LLMs evaluated in the paper · MMLU subset | PDR: -0.0102 | Hurt |
| Apply option format refactoring | LLMs evaluated in the paper · MMLU subset | PDR: -0.1151 | Hurt |
| Apply option ID shifting | LLMs evaluated in the paper · MMLU subset | PDR: -0.0059 | No clear effect |
| Convert questions to multi-judgment format | LLMs evaluated in the paper · MMLU subset | PDR: -0.1083 | Hurt |
| Use PertEval rather than PromptAttack to generate knowledge-invariant perturbations | benchmark question paraphrasing · C-Math, W-History, P-Psychology, P-Medicine | human knowledge-invariance score | Helped |
| Perturb the options by adding incorrect choices alongside a correct choice | gpt-4-turbo · MMLU subset, composite perturbation | multiple selection including the correct option: 12% → 21.7% | Hurt |
Surprising findings
- Changing only the global position of the question versus options (SwapPos) substantially lowers performance despite being knowledge-invariant.
- Edit distance (Levenshtein) shows no clear correlation with human/LLM knowledge-invariance scores.
Limitations
- Applicable only to close-ended benchmarks, not open-ended ones.
- Limited controllability of LLM-based paraphrasing.
- Experiments limited to a four-subject subset of MMLU due to budget.
Experimental setup
Datasets: MMLU college mathematics MMLU high school world history MMLU professional psychology MMLU professional medicine
Benchmarks: MMLU (C-Math, W-History, P-Psychology, P-Medicine subsets)
Metrics: ACC@Consist PDR ROP knowledge invariance score accuracy
Baselines: PromptAttack
Builds on: MMLU PromptAttack CheckList Wilcoxon signed-rank test
Closest papers by meaning
- KoLA: Carefully Benchmarking World Knowledge of Large Language Models (ICLR 2024 · similarity 0.90)
- metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.89)
- Measuring what Matters: Construct Validity in Large Language Model Benchmarks (NeurIPS 2025 · similarity 0.89)
- Brittlebench: Quantifying LLM robustness via prompt sensitivity (NeurIPS 2026 · similarity 0.89)
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations (ICLR 2025 · similarity 0.89)
- KScope: A Framework for Characterizing the Knowledge Status of Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.