Research map / Large language models / Language, safety and interpretability

Online Detection of LLM-Generated Texts via Sequential Hypothesis Testing by Betting

ICML 2025 · Large language models › Language, safety and interpretability · approach: Zero-shot machine-text detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an online LLM-text detector that turns offline score functions into a sequential betting test (ONS-updated wealth) with controlled false positive rate and bounded expected detection time.

Problem

Detecting whether a streaming text source is an LLM with anytime-valid statistical guarantees on false positives and bounded detection time, without threshold tuning; offline detectors cannot control type-I error online.

Approach

Frame online LLM detection as a two-sample sequential test: compare score differences between a human-text pool and the incoming stream, bet on the mean difference with Online Newton Step updates to grow wealth under the alternative, and reject when wealth exceeds 1/alpha (randomized Ville's inequality for a time budget). Extends testing-by-betting to unknown score ranges via estimated bounds d_t

Key results

• Under oracle knowledge of d_t and epsilon, configurations using Fast-DetectGPT, DetectGPT, or Likelihood detect LLM sources around t=100 on average while keeping FPR below alpha across 20 alpha levels.
• Under estimated parameters (Scenario 2), Fast-DetectGPT with Neo-2.7 still controls FPR and detects LLMs at roughly t=300 for most score functions, though RoBERTa-based and some Gemma-2B configurations do not strictly control FPR.
• Permutation-test baselines fail to control type-I error with small batch sizes and reject almost immediately after the first batch, while the proposed method keeps FPR below alpha.

Measured effects reported in the paper

Surprising findings

• Baseline permutation tests with corrected alpha/2^i still fail to be valid alpha-tests under H0 for small batch sizes, due to increased score variability.
• Some RoBERTa-based and Gemma-2B configurations violate FPR control when estimated d_t is too small, breaking the nonnegative supermartingale property.

Limitations

• Performance depends on the choice of score function and on accurately estimating score-range bound d_t and margin epsilon before testing.
• Power analysis assumes i.i.d. data; a good score function with differing means for human and LLM text is required for H0 to equal human source.
• Some configurations (RoBERTa-based, Gemma-2B combinations) fail to strictly control FPR when d_t is underestimated.

Experimental setup

Datasets: XSum Paris 2024 Olympics news (500 articles from official website) machine-generated news from Gemini-1.5-Flash, Gemini-1.5-Pro, PaLM 2

Benchmarks: XSum vs. 2024 Olympic news (H0) XSum vs. machine-generated Olympic news (H1)

Metrics: false positive rate rejection time (expected time to detect LLM)

Baselines: fixed-batch permutation test (constant alpha) fixed-batch permutation test (alpha/2^i)

Builds on: testing by betting Online Newton Step Ville's inequality Fast-DetectGPT DetectGPT

Closest papers by meaning

• Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.90)
• AdaDetectGPT: Adaptive Detection of LLM-Generated Text with Statistical Guarantees (NeurIPS 2025 · similarity 0.88)
• LLM Dataset Inference: Did you train on my dataset? (NeurIPS 2024 · similarity 0.87)
• Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection (NeurIPS 2025 · similarity 0.87)
• Can LLM-Generated Misinformation Be Detected? (ICLR 2024 · similarity 0.86)
• DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text Detection (NeurIPS 2024 · similarity 0.86)