Research map / Large language models / Language, safety and interpretability
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MAUVE to compare generated text with human text | GPT-2 XL decoding on web text · Web text | MAUVE: 0.02 -> 0.88 -> 0.94 | Helped |
| Increase GPT-2 model size | GPT-2 with nucleus sampling · Web text | MAUVE: 0.878 -> 0.915 -> 0.936 -> 0.940 | Helped |
| Increase maximum generation length | GPT-2 · Web text | MAUVE | Hurt |
| Replace standard decoding with adversarial perplexity sampling | GPT-2 · Web text | MAUVE: 0.06 | Hurt |
| Use MAUVE instead of generation perplexity for ranking generation quality | GPT-2 generations · Web text, human-like preference | Spearman correlation: 0.810 -> 0.952 | Helped |
| Use MAUVE instead of generation perplexity for ranking generation quality | GPT-2 generations · Web text, interestingness preference | Spearman correlation: 0.643 -> 0.810 | Helped |
| Use MAUVE instead of generation perplexity for ranking generation quality | GPT-2 generations · Web text, sensible preference | Spearman correlation: 0.738 -> 0.857 | Helped |
| Use MAUVE to rank generation quality | News generations · News | Discriminator accuracy correlation: 0.792 -> 0.956 | Helped |
| Use MAUVE to rank generation quality | Story generations · Stories | Discriminator accuracy correlation: 0.857 -> 0.893 | Helped |
| Replace GPT-2 Large embeddings with RoBERTa Large embeddings (ablation) | MAUVE · Web text, model sizes and decoding algorithms | Spearman rank correlation: 0.993 | No clear effect |
| Vary k-means cluster count from 100 to 5000 (ablation) | MAUVE · Web text, model sizes and decoding algorithms | Spearman rank correlation: 0.99 or 1.00 | No clear effect |
| Replace k-means with DRMM or lattice quantization (ablation) | MAUVE · Web text, model sizes and decoding algorithms | Spearman rank correlation: at least 0.99 | No clear effect |
Closest papers by meaning
- Open-Domain Text Evaluation via Contrastive Distribution Methods (ICML 2024 · similarity 0.87)
- Compressed and Smooth Latent Space for Text Diffusion Modeling (NeurIPS 2025 · similarity 0.86)
- OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation (CVPR 2025 · similarity 0.86)
- AI as Humanity’s Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text (ICLR 2025 · similarity 0.86)
- Correlation Dimension of Autoregressive Large Language Models (NeurIPS 2025 · similarity 0.86)
- Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond) (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.