Research map / Large language models / Language, safety and interpretability
Telescope: Improving Zero Shot Detection of LLM Generated Content By Measuring Token Repetition Probability
Christopher Nassif, Joshua F. Cooper
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use only bigram context instead of full context (ablation) | Telescope Perplexity · HC3 | AUC: 0.995 -> 0.897 | Hurt |
| use only bigram context instead of full context (ablation) | Perplexity · HC3 | AUC: 0.991 -> 0.761 | Hurt |
| use trigram context instead of full context (ablation) | Telescope Perplexity · HC3 | AUC: 0.995 -> 0.921 | Hurt |
| score the current token given itself (Telescope) instead of next-token perplexity | SmolLM 360M detector · Detect LLM Text | AUROC: 0.89307 -> 0.99219 | Improved |
| score the current token (Telescope) instead of next-token perplexity | SmolLM 360M detector · ESL GPT4o Mini | AUROC: 0.82523 -> 0.99983 | Improved |
| Telescope instead of Binoculars | zero-shot detector · GB News ChatGPT | AUROC: 0.98773 -> 0.90480 | Hurt |
| Telescope instead of Perplexity | zero-shot detector · GB Essay ChatGPT | AUROC: 0.99810 -> 0.98628 | Hurt |
| Telescope instead of Binoculars | zero-shot detector · HC3 Plus | AUROC: 0.88510 -> 0.98451 | Improved |
| raise target sampling temperature from 0 to 1.2 (ablation) | Telescope Perplexity · GB Essay GPT-4o | AUROC: 0.999992 -> 0.9988 | Hurt |
| adversarial prompt asking target model to repeat key words (ablation) | Telescope Perplexity · GB Essay GPT-4o | AUROC: 0.99993 -> 0.996 | Hurt |
| evaluate Telescope on its own training data split instead of Perplexity | SmolLM 360M · FineWeb vs Cosmopedia | AUC: 0.9523 -> 0.9956 | Improved |
| Telescope instead of Binoculars on Russian text | Gemma2-2B · M4 Russian | AUC: 0.89772 -> 0.66543 | Hurt |
Closest papers by meaning
- Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs (ICLR 2025 · similarity 0.87)
- Log-Likelihood, Simpson’s Paradox, and the Detection of Machine-Generated Text (NeurIPS 2026 · similarity 0.87)
- Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text (ICML 2024 · similarity 0.87)
- Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (ICLR 2024 · similarity 0.87)
- Zero-Shot Detection of LLM-Generated Text via Implicit Reward Model (NeurIPS 2025 · similarity 0.86)
- Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
