Research map / Large language models / Language, safety and interpretability
Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MMD-MP, a multi-population-aware optimization of deep-kernel MMD that drops the intra-MGT term from the objective to reduce variance, and applies it to paragraph-level two-sample tests and sentence-level single-instance detection of machine-generated text.
Problem
Detecting machine-generated texts (MGTs) from many LLMs is hard because the MGT distribution is a mixture of several populations, so training a deep-kernel MMD detector yields high-variance, unstable discrepancy estimates and low test power.
Approach
Optimize kernel-based MMD with a multi-population proxy (MPP) that excludes the intra-class distance among MGTs, so the kernel need not aggregate all heterogeneous MGTs; this lowers variance and improves transfer to unseen LLMs.
Key results
- On HC3 with 1,000 training paragraphs, MMD-MP reaches 92.31 test power on ChatGPT vs 91.38 for MMD-D, and 79.92 vs 62.34 on GPT2-S/GPT2-M mixed MGTs (ChatGPT+GPT2-S+GPT2-M), with reported gains of 8.20% (two populations) and 13.97% (three populations) over MMD-D on average.
- On unknown LLMs (GPT-Neo-L, GPT-j-6b, GPT4all-j), MMD-MP attains test power of 61.79, 59.57 and 77.69, improving over MMD-D by 23.61 to 27.65 points.
- Sentence-level AUROC on HC3 improves over MMD-D by 0.22% to 1.71% and over CE-Classifier by 0.61% to 2.64% on full training data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-population-aware MMD-MP optimization instead of MMD-D | MGT paragraph detection · HC3, 1,000 training paragraphs; multiple LLM populations | test power/100: MMD-MP outperforms MMD-D by an average of 8.20% on test power over the two training populations and 13.97% over the thre | Helped |
| Use multi-population-aware MMD-MP optimization instead of MMD-D | MGT paragraph detection · Unknown LLMs: Neo-L, GPT-j-6b, GPT4all-j | test power/100: 23.61%{[0m\sim}27.65% {\uparrow} | Helped |
| Use multi-population-aware MMD-MP optimization instead of MMD-D | MGT sentence detection · Unknown LLMs: Neo-L, GPT-j-6b, GPT4all-j | AUROC/100: 3.25% \uparrow | Helped |
| Exclude the MGT intra-class distance during kernel optimization | MMD-MP · Multiple-population MGT detection | variance of MMD | Helped |
| Use MMD rather than MPP to measure the test discrepancy | MMD-MP paragraph detection · HC3, paragraph-based detection | detection performance | No clear effect |
Surprising findings
- Excluding the intra-MGT term from the objective, and using MMD rather than MPP at test time, is reported to yield almost identical performance to the alternative test-time strategy.
- MMD-O (Gaussian kernel with optimized bandwidth) performs near chance (test power around 20-27) on HC3 paragraphs.
Limitations
- Gains are reported on HC3 and XSum with a small set of LLMs; generalization to other domains is not fully characterized.
- Theoretical analysis relies on assumptions (bounded kernels, Lipschitz parameterization) and an epsilon-net argument; practical benefit depends on empirical evidence.
- Results are averaged over 10 or 5 repeats, with substantial standard deviations on some settings.
Experimental setup
Datasets: HC3 XSum
Benchmarks: HC3 XSum
Metrics: test power AUROC
Baselines: MMD-O MMD-D C2ST-S C2ST-L CE-Classifier Log-Likelihood Entropy Rank Log-Rank DetectGPT OpenAI-D ChatGPT-D
Builds on: MMD-D (deep kernel MMD) C2ST Test power optimization of MMD (Liu et al.) DetectGPT
Closest papers by meaning
- DALD: Improving Logits-based Detector without Logits from Black-box LLMs (NeurIPS 2024 · similarity 0.89)
- Position: On the Possibilities of AI-Generated Text Detection (ICML 2024 · similarity 0.88)
- Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection (NeurIPS 2025 · similarity 0.88)
- Imitate Before Detect: Aligning Machine Stylistic Preference for Machine-Revised Text Detection (AAAI 2025 · similarity 0.88)
- Alignment Imprint: Zero-Shot AI-Generated Text Detection via Provable Preference Discrepancy (NeurIPS 2026 · similarity 0.87)
- Training-free LLM-generated Text Detection by Mining Token Probability Sequences (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.