Research map / Large language models / AI and society
Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a maximum-likelihood, corpus-level estimator of the fraction of LLM-modified text (using adjective occurrence distributions from human vs. GPT-generated reviews) and applies it to ICLR, NeurIPS, CoRL, EMNLP reviews after ChatGPT's release.
Problem
Instance-level AI-text detectors are unreliable, so there is no precise way to measure how much of a real-world corpus (here, peer reviews) has been substantially modified or generated by LLMs.
Approach
Estimate the mixing fraction alpha in a mixture of human and AI text distributions via maximum likelihood, using token (adjective) occurrence probabilities estimated from known human-written and LLM-generated reference corpora, avoiding per-document classification.
Key results
- Estimated AI-modified fraction rose from 1.6% to 10.6% for ICLR (2023 to 2024) and from 1.9% to 9.1% for NeurIPS (2022 to 2023), with EMNLP 2023 at about 16.9%.
- Validation on semi-synthetic mixtures gives prediction error under 1.8% on ICLR 2023 and NeurIPS 2022, and under 2.4% on CoRL 2022.
- Compared with a BERT classifier baseline, the method reduces in-distribution estimation error 3.4x (6.2% to 1.8%) and OOD error 4.6x (11.2% to 2.4%), at over 10 million times lower inference FLOPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply the estimator to post-ChatGPT conference reviews rather than pre-ChatGPT reviews | ML conference reviews · ICLR | estimated LLM-generated text fraction (α): 1.6% -> 10.6% | Helped |
| Apply the estimator to post-ChatGPT conference reviews rather than pre-ChatGPT reviews | ML conference reviews · NeurIPS | estimated LLM-generated text fraction (α): 1.9% -> 9.1% | Helped |
| Apply the estimator to post-ChatGPT conference reviews rather than pre-ChatGPT reviews | ML conference reviews · CoRL | estimated LLM-generated text fraction (α): 2.4% -> 6.5% | Helped |
| Use distributional GPT quantification instead of the best-performing instance-based baseline | AI text fraction estimation · In-distribution validation | estimation error: 6.2% -> 1.8% | Helped |
| Use distributional GPT quantification instead of the best-performing instance-based baseline | AI text fraction estimation · Out-of-distribution validation | estimation error: 11.2% -> 2.4% | Helped |
| Use adjectives rather than adverbs as the token vocabulary (ablation) | Distributional GPT quantification · Semi-synthetic validation | prediction error | Helped |
| Use sentences rather than documents as the estimation unit (ablation) | Distributional GPT quantification · Validation | estimation performance | Helped |
| Train with one prompt and test with a different prompt (ablation) | Distributional GPT quantification · ICLR 2023, NeurIPS 2022, and CoRL 2022 validation | prediction error | No clear effect |
| Use ChatGPT for proofreading only, limited to typos and grammar (ablation) | Peer reviews · Proofreading robustness check | estimated LLM-generated text fraction (α) | No clear effect |
| Use an LLM to expand a human-written review outline into a full review (ablation) | Peer reviews · Semi-synthetic validation | estimated LLM-generated text fraction (α) | Helped |
| Compare reviews submitted within 3 days of the deadline with other reviews | Conference peer reviews · Deadline stratification | estimated LLM-generated text fraction (α) | Helped |
| Compare low-confidence reviews (confidence 2 or lower) with reviews rated 3 or above | Conference peer reviews · Confidence stratification | estimated LLM-generated text fraction (α) | Helped |
Surprising findings
- Nature portfolio journals showed no significant increase in estimated AI-modified text after ChatGPT's release, unlike ML conferences.
- Proofreading with ChatGPT produced only a minor, non-significant increase in estimated alpha, suggesting the method targets substantive edits rather than grammar fixes.
Limitations
- Estimates are population-level and cannot classify individual reviews; results depend on the chosen LLM and prompts used to generate reference text.
- Assumes token occurrences are independent and relies on stable token frequencies across time, which may shift with topics and reviewers.
- Does not establish that LLMs wrote reviews from scratch; the estimate reflects substantial modification, and effects like the deadline or reply-rate correlations are not causal.
Experimental setup
Datasets: ICLR 2018-2022 official reviews ICLR 2023 reviews NeurIPS 2017-2022 reviews CoRL 2021-2022 reviews ICLR 2024 reviews NeurIPS 2023 reviews CoRL 2023 reviews EMNLP 2023 reviews Nature portfolio reviews
Benchmarks: ICLR 2023 semi-synthetic validation NeurIPS 2022 OOD validation CoRL 2022 OOD validation Nature portfolio 2022 validation
Metrics: estimated alpha prediction error confidence interval computational FLOPS
Baselines: BERT classifier DetectGPT Fast-DetectGPT
Builds on: DetectGPT Fast-DetectGPT
Closest papers by meaning
- AAAR-1.0: Assessing AI’s Potential to Assist Research (ICML 2025 · similarity 0.87)
- From Replication to Redesign: Exploring Pairwise Comparisons for LLM-Based Peer Review (NeurIPS 2025 · similarity 0.86)
- RevisEval: Improving LLM-as-a-Judge via Response-Adapted References (ICLR 2025 · similarity 0.86)
- Benchmarking LLMs' Judgments with No Gold Standard (ICLR 2025 · similarity 0.86)
- Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models (ICLR 2024 · similarity 0.86)
- CycleResearcher: Improving Automated Research via Automated Review (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.