Research map / Large language models / Language, safety and interpretability
Position: On the Possibilities of AI-Generated Text Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper argues theoretically via Le Cam's lemma and Chernoff information that AI-generated text is detectable whenever human and machine distributions differ, with sample complexity bounds for iid and non-iid settings, and validates with experiments on XSum, Squad, IMDb and Kaggle Fake News.
Problem
Prior impossibility results (Sadasivan et al., 2023) suggest AI-generated text detection is infeasible when human and machine text distributions have small total variation distance; the question is whether detection is possible in practice given enough text.
Approach
Shows that the total variation between product distributions over n samples (or longer sequences) approaches 1 exponentially fast in n via Chernoff information, so the AUROC upper bound of the best likelihood-ratio detector grows with the number of samples; derives iid and non-iid sample complexity bounds that make detection possible for any nonzero distribution gap.
Key results
- Best-possible (likelihood-ratio) detector AUROC on XSum and Squad rises from about 58% to 97% as n-gram length goes from 1 to 6.
- Zero-shot RoBERTa and GPTZero detection AUROC improves from around 50% to about 90% as sample length increases on XSum and Squad.
- Pairwise iid samples raise TF-IDF logistic regression AUROC from about 73% to 97%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase n-gram feature length from 1 to 6 (ablation) | Best possible detector · XSum | AUROC: 58% -> 97% | Helped |
| Increase n-gram feature length from 1 to 6 (ablation) | Best possible detector · Squad | AUROC: 58% -> 97% | Helped |
| Use 2 IID samples instead of one (ablation) | Vanilla Logistic Regression detector · Human-machine classification datasets | AUROC: 73% to 97% | Helped |
| Increase the number of samples or sequence length (ablation) | Zero-shot detectors (RoBERTa-Large, RoBERTa-Base, GPTZero) · XSum and Squad | AUROC: around 50% to 90% | Helped |
| Increase sequence length (ablation) | Logistic Regression, Random Forest, and MLP detectors · XSum and Squad | AUROC | Helped |
| Apply paraphrasing attacks | AI-generated text detectors · AI-generated text detection | detection performance | Hurt |
Surprising findings
- Detection of machine text is feasible for any nonzero total variation gap, contradicting the pessimistic conclusion drawn from small TV bounds in prior work.
- Word-level IMDb best AUROC is only about 0.585, yet paragraph-level real detectors exceed 0.85 train AUROC.
Limitations
- Theory assumes i.i.d. or a simple dependence structure (rho-based) that may not hold for real text, and the bounds are order-notation rather than tight constants.
- Experiments use older generators (GPT-2, Llama family) with TF-IDF classifiers and zero-shot detectors on a limited number of 500 passages; paraphrasing results are only in an appendix.
- Practical detectors that process many samples are more complex, and the paper does not provide a new detector.
Experimental setup
Datasets: XSum Squad IMDb Kaggle Fake News
Benchmarks: XSum Squad IMDb Kaggle Fake News
Metrics: AUROC TV distance
Baselines: RoBERTa-Base-Detector RoBERTa-Large-Detector GPTZero Logistic Regression Random Forest MLP
Builds on: Sadasivan et al. 2023 impossibility result DetectGPT Le Cam's lemma Neyman-Pearson lemma watermarking (Kirchenbauer et al. 2023)
Closest papers by meaning
- Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection (NeurIPS 2025 · similarity 0.91)
- Multiscale Positive-Unlabeled Detection of AI-Generated Texts (ICLR 2024 · similarity 0.90)
- Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.90)
- DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning (NeurIPS 2024 · similarity 0.89)
- DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm (NeurIPS 2025 · similarity 0.89)
- Raidar: geneRative AI Detection viA Rewriting (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.