Research map / Large language models / Language, safety and interpretability
Zero-shot machine-text detection
Detect LLM-generated text with perplexity-based and reward-based zero-shot scores.
82 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in Language, safety and interpretability, keyword cluster “watermarking · generated text · watermarks”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
28 papers in 2023–24 and 46 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 4.0% to 2.5%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Exploring the Terrain of Metaphor Novelty: A Regression-Based Approach for Automatically Scoring Metaphors (AAAI 2018 · 11 citations)
- 2019 · Exploring Human-Like Reading Strategy for Abstractive Text Summarization (AAAI 2019)
- 2020 · Automatically Neutralizing Subjective Bias in Text (AAAI 2020 · 153 citations)
- 2021 · MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers (NeurIPS 2021 · 23 citations)
- 2022 · A Contrastive Framework for Neural Text Generation (NeurIPS 2022 · 83 citations)
- 2023 · Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense (NeurIPS 2023 · 90 citations)
- 2024 · Position: Will we run out of data? Limits of LLM scaling based on human-generated data (ICML 2024 · 83 citations)
- 2025 · Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting (AAAI 2025 · 11 citations)
- 2026 · EditLens: Quantifying the Extent of AI Editing in Text (ICLR 2026 · 3 citations)
Earliest papers here
- Exploring the Terrain of Metaphor Novelty: A Regression-Based Approach for Automatically Scoring Metaphors (AAAI 2018 · 11 citations)
- Exploring Human-Like Reading Strategy for Abstractive Text Summarization (AAAI 2019)
- Automatically Neutralizing Subjective Bias in Text (AAAI 2020 · 153 citations)
What has been tried
51 papers here report 518 measured changes (a component added, removed or swapped, compared with a baseline): helped 65%, no clear effect 7%, hurt 24%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Increase n-gram feature length from 1 to 6 | XSum · AUROC | 58% -> 97% | Helped | Position: On the Possibilities of AI-Generated Text Detection (ICML 2024 · 53 citations) |
| use GPT2-xl instead of text-davinci-003 for perplexity calculation | six programming languages · AUC | C: 0.79; C++: 0.75; C#: 0.73; Java: 0.72; JavaScript: 0.71; Python: 0.69 | Hurt | Detecting AI-Generated Code Assignments Using Perplexity of Large Language Models (AAAI 2024 · 45 citations) |
| Use GPT-J rather than the source model as the sampling model | XSum, SQuAD, and WritingPrompts; three source models · AUROC | +0.0020 AUROC; relative improvement of 27% | Helped | Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (ICLR 2024 · 20 citations) |
| Set γ=0.5 rather than γ=0.7 | Reddit-ELI5 · TPR at 1% FPR | 87.34 -> 91.00 | Helped | DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text (ICLR 2024 · 20 citations) |
| increase prototype size from p=1 to p=2 | In-Domain, All · F1@3 | 0.571 -> 0.575 | Helped | Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education (AAAI 2024 · 16 citations) |
| Combine perplexity and cross-perplexity as the Binoculars score instead of using Falcon perplexity alone | Writing Prompts · TPR @ 0.01% FPR | 0.86 -> 0.93 | Helped | Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (ICML 2024 · 16 citations) |
| omit short training texts from backward operations | HC3-En-Sent · F1 | 58.60 ± 10.53 -> 62.42 ± 5.60 | Helped | Multiscale Positive-Unlabeled Detection of AI-Generated Texts (ICLR 2024 · 13 citations) |
| Use concatenation instead of absolute difference or dot product to combine feature vectors | Writing dataset generated by ChatGPT · AUROC | 89.81% and 93.84% ... 98.21% | Helped | DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection (NeurIPS 2024 · 12 citations) |
| Use code rewriting with similarity measurement rather than in-fill perturbation with similarity | APPS · AUROC | 74.55 -> 87.77; 71.55 -> 76.96; 74.23 -> 80.23; 53.43 -> 74.51 | Helped | Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting (AAAI 2025 · 11 citations) |
| Use multi-level contrastive loss instead of plain contrastive loss | Deepfake Cross-domains & Cross-models · AvgRec | 94.22 -> 96.15 | Helped | DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning (NeurIPS 2024 · 11 citations) |
| Train UAR with Reddit, Twitter, and StackExchange rather than Reddit alone | Unseen domains, topics, and LLM, N=5 · pAUC | 0.868 (0.001) -> 0.886 (0.001) | Helped | Few-Shot Detection of Machine-Generated Text using Style Representations (ICLR 2024 · 6 citations) |
| Use equivariance instead of invariance | News · F1 | 60.29 -> 58.00 | Hurt | Raidar: geneRative AI Detection viA Rewriting (ICLR 2024 · 5 citations) |
What reviewers will expect
Datasets and baselines named in the 26 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- XSum · 9 of 26 papers
- WritingPrompts · 7 of 26 papers
- PubMedQA · 4 of 26 papers
- DetectRL · 4 of 26 papers
- RAID · 4 of 26 papers
- M4 · 4 of 26 papers
- SQuAD · 3 of 26 papers
- MT-Bench · 2 of 26 papers
Baselines
- Fast-DetectGPT · 11 of 26 papers
- LogRank · 10 of 26 papers
- DetectGPT · 10 of 26 papers
- Binoculars · 8 of 26 papers
- Likelihood · 7 of 26 papers
- Entropy · 7 of 26 papers
- DNA-GPT · 6 of 26 papers
- LRR · 5 of 26 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 74 papers; from OpenAlex and the papers' affiliations).
- Carnegie Mellon University · 4 papers
- MBZUAI · 3 papers
- University of Pennsylvania · 2 papers
- Texas Tech University · 2 papers
- Stanford University · 2 papers
- South China University of Technology · 2 papers
- The Hong Kong Polytechnic University · 2 papers
- University of California, San Diego · 2 papers
- Google · 2 papers
- Huawei · 2 papers
Use it
Download BibTeX of 30 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Text watermarking (80 papers)
- Data and hallucination detection (66 papers)
Elsewhere in the topic
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- Uncertainty quantification for LLMs (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Statistical frameworks for LLM evaluation (124 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).