Research map / Large language models / Language, safety and interpretability
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, Chelsea Finn
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace log-prob thresholding with perturbation discrepancy (DetectGPT, T5-3B perturbations) | zero-shot detection of model samples · XSum, SQuAD, WritingPrompts (Table 1) | AUROC: +0.10 avg XSum, +0.05 SQuAD, +0.01 WritingPrompts over strongest baseline | Improved |
| use perturbations from mask-filling model (T5) rather than no perturbation (log p baseline) | GPT-3 and Jurassic-2 detection · PubMedQA, XSum, WritingPrompts (Table 2) | average AUROC: 0.69 to 0.83 | Improved |
| nucleus (top-p=0.96) or top-k=40 sampling for generation (ablation) | all zero-shot detectors · XSum, SQuAD, WritingPrompts (Table 3 and appendix) | AUROC: detection easier on average for all methods | Improved |
| increase number of perturbations from few to 100 (ablation) | DetectGPT · GPT-2 and GPT-J on three datasets | AUROC | Improved |
| use a small mask-filling model (random or smaller T5) instead of larger T5 (ablation) | DetectGPT · SQuAD contexts | AUROC | Hurt |
| score with a surrogate model different from the source model (ablation) | DetectGPT · XSum, SQuAD, WritingPrompts (GPT-J, Neo-2.7, GPT-2) | AUROC | Hurt |
| replace a fraction of model text with T5 edits (human-edit simulation) (ablation) | DetectGPT · XSum paraphrase experiment | AUROC | Hurt |
| normalize perturbation discrepancy by std of perturbed log-probs (ablation) | DetectGPT · main experiments | AUROC: around 0.020 increase | Improved |
| apply all T5 mask perturbations at once on very long passages (ablation) | DetectGPT · passage length analysis | AUROC | Hurt |
| use supervised RoBERTa detectors instead of DetectGPT on out-of-distribution domains (PubMed, German) | supervised detectors · PubMedQA and WMT16 German | AUROC | Hurt |
Later papers that compared with this method
7 later accepted papers report DetectGPT as a baseline in their results tables; 7 of them report a better number on most of the results they share with it.
- Beyond Raw Detection Scores: Markov-Informed Calibration for Boosting Machine-Generated Text Detection (ICLR 2026 · better on 2 of 2 shared results)
- On the Salience of Low-Probability Tokens for AI-Generated Text Detection: A Multiscale Uncertainty Perspective (ICML 2026 · better on 1 of 1 shared result)
- SpecDetect: Simple, Fast, and Training-Free Detection of LLM-Generated Text via Spectral Analysis (AAAI 2026 · better on 1 of 1 shared result)
- Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (ICLR 2024 · better on 4 of 4 shared results)
- Multiscale Positive-Unlabeled Detection of AI-Generated Texts (ICLR 2024 · better on 2 of 2 shared results)
- Raidar: geneRative AI Detection viA Rewriting (ICLR 2024 · better on 1 of 1 shared result)
- Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (NeurIPS 2023 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (ICLR 2024 · similarity 0.94)
- Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection (ICLR 2025 · similarity 0.89)
- DALD: Improving Logits-based Detector without Logits from Black-box LLMs (NeurIPS 2024 · similarity 0.88)
- Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.88)
- Zero-Shot Detection of LLM-Generated Text via Implicit Reward Model (NeurIPS 2025 · similarity 0.87)
- DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
