Research map / Label-efficient and robust learning / OOD and anomaly detection
Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reframes LLM-generated text detection as one-class out-of-distribution detection, treating human text as outliers, and trains DeepSVDD, HRN, or energy-based heads on top of a text encoder to reach SoTA on DeepFake, M4, and RAID.
Problem
Binary classifiers for LLM-generated text detection assume human text is one coherent distribution, which leads to poor generalization across domains, models, and attacks.
Approach
Model only the machine-generated text as the in-distribution class with one-class (DeepSVDD, HRN) or energy-based OOD losses, treating diverse human-written text as outliers, combined with a SimCLR-style contrastive loss.
Key results
- On DeepFake, the DeepSVDD variant reaches 98.3 AUROC and AUPR with 8.9% FPR95, versus 95.6 AUROC and 16.1% FPR95 for DeTeCtive.
- On M4-multilingual, DeepSVDD and HRN reach 96.4 AUROC against DeTeCtive's 93.1, and the energy variant gets 10.7% FPR95 against 52.4%.
- On RAID, the energy variant reaches 99.7 AUPR and 28.3% FPR95, while DeepSVDD reaches 94.7 AUROC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reframe detection as OOD detection, modeling generated text as ID and human text as OOD, rather than use binary classification (ablation) | LLM text detection · DeepFake | AUROC: 89.0 -> 98.3 | Helped |
| reframe detection as OOD detection rather than use binary classification (ablation) | LLM text detection · M4-multilingual | AUROC: 84.6 -> 96.4 | Helped |
| reframe detection as OOD detection rather than use binary classification (ablation) | LLM text detection · RAID attacked | AUROC: 84.5 -> 94.7 | Helped |
| add D-SVDD loss instead of the binary classification head (ablation) | LLM text detection · DeepFake | AUROC: 89.0 -> 98.3 | Helped |
| add HRN loss instead of the binary classification head (ablation) | LLM text detection · M4-multilingual | AUROC: 84.6 -> 96.4 | Helped |
| add Energy loss instead of the binary classification head (ablation) | LLM text detection · RAID attacked | AUROC: 84.5 -> 93.6 | Helped |
| treat human-written text as OOD rather than LLM-generated text (ablation) | DeepSVDD · DeepFake | AUROC: 69.6 -> 93.8 | Helped |
| add contrastive loss (β=1 rather than β=0) (ablation) | DeepSVDD on DeepFake · DeepFake | AUROC: 88.6 -> 98.3 | Helped |
| add OOD loss (α=1 rather than α=0) (ablation) | DeepSVDD on DeepFake · DeepFake | AUROC: 93.7 -> 98.3 | Helped |
| use BERT_large instead of SimCSE-RoBERTa_base as the backbone (ablation) | DeepSVDD · DeepFake | AUROC: 98.3 -> 93.1 | Hurt |
| use Energy loss rather than D-SVDD loss (ablation) | OOD detector · RAID attacked | AUPR: 73.0 -> 99.7 | Helped |
| use HRN rather than D-SVDD | OOD detector · DeepFake unseen domain | AUROC: 97.9 -> 98.0 | No clear effect |
Surprising findings
- Modeling human text as the OOD class outperforms modeling LLM text as OOD (93.8 vs 69.6 AUROC in the ablation table).
- The classification-head ablation shows FPR95 of 75.8-83.5 across benchmarks, far above the one-class losses.
Limitations
- Some baselines were evaluated on random 10K subsets because of slow perturbation-based detection, which limits comparability.
- The theoretical argument relies on approximate assumptions about human text distributions, such as a shifted biased training distribution.
- Results on newer models are limited to a RAID subset and a GPT-4o XSum set.
Experimental setup
Datasets: DeepFake M4 RAID EvoBench
Benchmarks: DeepFake M4-multilingual RAID EvoBench
Metrics: AUROC AUPR FPR95 accuracy F1
Baselines: DetectLLM DetectGPT DNA-GPT Fast-DetectGPT Binoculars DALD Glimpse GPTZero RADAR GhostBuster BiScope DeTeCtive
Builds on: DeepSVDD HRN energy-based OOD detection DeTeCtive SimCLR
Closest papers by meaning
- Position: On the Possibilities of AI-Generated Text Detection (ICML 2024 · similarity 0.91)
- DALD: Improving Logits-based Detector without Logits from Black-box LLMs (NeurIPS 2024 · similarity 0.90)
- Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.89)
- Few-Shot Detection of Machine-Generated Text using Style Representations (ICLR 2024 · similarity 0.89)
- Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection (ICLR 2025 · similarity 0.88)
- OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.