AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Hubble: a Model Suite to Advance the Study of LLM Memorization

Johnny Wei, Ameya Godbole, Mohammad Aflah Khan, Ryan Yixiang Wang, Xiaoyuan Zhu, James Flemings, Nitya Kashyap, Krishna P. Gummadi, Willie Neiswanger, Robin Jia

ICLR 2026 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train on larger corpus (500B vs 100B tokens) with same inserted data (dilution) (ablation)Hubble 8B perturbed · memorization across copyright/privacy/test-set tasksmemorization vs duplicate countImproved
insert sensitive data only in first quarter of training (ablation)Hubble 1B timing runs · memorization evalsmemorizationImproved
insert perturbations only in last quarter of training (ablation)Hubble 1B timing runs · memorization evalsmemorizationHurt
increase model size from 1B to 8B (ablation)Hubble perturbed (500B) · memorization evalsmemorization at fixed duplicationHurt
mix perturbations from all three domains vs single domain (ablation)Hubble 1B 100B perturbed · memorization evalsmemorizationNo clear effect
use popular rather than unpopular Gutenberg books (ablation)Hubble 1B/8B · passage memorizationgenerative extractionNo clear effect
give attacker more auxiliary information (full prefix vs name only) (ablation)PII attacks on Hubble 8B 100B · YAGO/ECtHR PII inferenceattack accuracy: close to 100% at 16 duplicationsImproved
contaminate test sets with inserted examples (ablation)Hubble perturbed models · unseen test examples (WinoGrande, PopQA, HellaSwag)accuracy at 0 duplicatesMixed
increase duplicate count of inserted examples (ablation)MIA methods on Hubble 8B 500B · Gutenberg UnpopularROC AUC: 0.539 -> 1.0 (Loss, dup 1 -> 256)Improved
use MinK%++ instead of Loss at high duplication (ablation)MIA on Gutenberg Unpopular dup=256 · Hubble 8B 500BROC AUC: 1.0 -> 0.949Hurt
use MinK%++ instead of Loss at low-medium duplication (ablation)MIA on MMLU dup=16 · Hubble 8B 500BROC AUC: 0.857 -> 0.943Improved
apply unlearning (RMU, RR, SatImp)Hubble 8B 500B perturbed · Gutenberg Unpopular / YAGOunlearn vs keep setMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy