AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Death of the Novel(ty): Beyond N-Gram Novelty as a Metric for Textual Creativity

Arkadiy Saakyan, Najoung Kim, Smaranda Muresan, Tuhin Chakrabarty

ICLR 2026 · Large language models › Language, safety and interpretability · approach: Zero-shot machine-text detection

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
higher n-gram novelty (log-standardized perplexity) of an expressionexpression-level creativity judgments · Expert close reading (OLMo, OLMo-2, human)odds of judged creative: OR 1.96 per SDImproved
use n-gram novelty alone as a creativity filter (top quartile)n-gram novelty metric · Expert close readingshare judged creative: approx 91% of top-quartile novel expressions not judged creativeHurt
higher n-gram novelty in OLMo-2 generated textOLMo-2 · Expert close readingpragmaticality: beta -0.48Hurt
higher n-gram novelty in OLMo generated textOLMo · Expert close readingpragmaticality: beta -0.17Hurt
higher n-gram novelty in human-written texthuman text · Expert close readingpragmaticality: beta 0.01No clear effect
switch from human to LLM authorship (GPT-5, Claude 4.1, OLMo, OLMo-2)passage generation source · Expert close readingodds of expression judged creative: OR 0.511 (GPT-5), 0.521 (Claude-4.1) vs humanHurt
higher passage-level AI-detector likelihoodpassage · Expert close readingcreativity: beta 0.05, p = 0.624No clear effect
higher writing quality reward model scorepassage · Expert close readingcreativity: OR 1.30Improved
few-shot prompting instead of zero-shot (ablation)LLM-as-a-Judge close reading · Novel / non-pragmatic expression identificationF1No clear effect
finetune smaller models on 60% of datasetOLMo2 7B, Qwen3 8B, Llama-3.1 8B · Close reading taskF1Hurt
use LLM-J novelty score differences to predict expert preferenceGPT-5 few-shot · Style Mimic expert preferencepreference coefficient: beta 0.63, OR 1.88Improved
use LLM-J pragmaticality score differences to predict expert preferenceGPT-5 few-shot · Style Mimic expert preferencepreference coefficient: beta -0.05, p = 0.832No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy