AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Are Language Models Any Good at Density Modeling?

Sriram Ranga, Sai Shashank Bedampeta, Rui Mao, Anupam Chattopadhyay

AAAI 2026 · Large language models › Language, safety and interpretability · approach: LLM unlearning and benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train on uniform distribution over numbersGPT-2-style LM · UniformKL divergence: 0.2282 -> 0.0038Improved
train on linear distributionGPT-2-style LM · LinearKL divergence: 0.3874 -> 0.0290Improved
train on binomial distributionGPT-2-style LM · BinomialKL divergence: 82.0749 -> 24.8572Improved
train on sine distribution with period 10 (syntactic suffix cue)GPT-2-style LM · Sine period 10KL divergence: 0.4274 -> 0.0070Improved
train on sine distribution with period 3 (needs semantic divisibility)GPT-2-style LM · Sine period 3KL divergence: 0.6880 -> 0.7323Hurt
train on divisibility-by-3 distributionGPT-2-style LM · Divisibility by 3 patterndistribution fitHurt
increase fraction of language used for training while keeping steps fixedGPT-2-style LM · Syntax error rateincorrectly formed numbersNo clear effect
train on 10% of the numbers instead of more dataGPT-2-style LM · Syntax error rateincorrectly formed numbers: less than 1%No clear effect
numbers frequently appearing as substrings of other numbersGPT-2-style LM trained on uniform · Probability of numbers one to one hundredassigned probabilityHurt
train in passage mode with multiple numbers per passageGPT-2-style LM · First-number distributiondistribution fitNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy