AI Paper Atlas

Research map / Probabilistic and scientific modeling / Molecules and proteins

Learning a Generative Meta-Model of LLM Activations

Grace Luo, Jiahai Feng, Trevor Darrell, Alec Radford, Jacob Steinhardt

ICML 2026 · Probabilistic and scientific modeling › Molecules and proteins · approach: Neuro-symbolic probabilistic layers · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
scale GLP from 3 to 24 layers (0.5B to 3.3B) (ablation)GLP on Llama1B · Frechet distanceFD: 0.68 -> 0.53Improved
use GLP instead of SAE reconstructionLlama8B activations · Frechet distanceFD: 6.91 -> 5.93Improved
use GLP instead of SAE reconstructionLlama8B-Base · Delta LM Lossdelta LM loss: 0.1976 -> 0.0513Improved
transfer GLP trained on Base to Instruct (ablation)GLP · Delta LM Lossdelta LM loss: 0.0513 -> 0.0860Hurt
use GLP meta-neurons instead of raw MLP neuronsLlama8B · 1-D probingprobe AUC: 0.82 -> 0.87Improved
use GLP meta-neurons instead of SAELlama1B · 1-D probingprobe AUC: 0.70 -> 0.84Improved
use GLP features instead of raw MLP neuronsLlama8B · Dense probingprobe AUC: 0.94 -> 0.94No clear effect
use GLP instead of SAELlama1B · Dense probingprobe AUC: 0.85 -> 0.92Improved
train one multi-layer GLP on all layers instead of single-layer (ablation)GLP on Llama1B · Frechet distanceFD: 0.55 -> 0.66Hurt
post-process steering with GLP diffusion projection (t_start 0.5, 20 steps)DiffMean, Persona Vector, SAE steering · Steering concept vs fluencyPareto frontierImproved
scale GLP compute (ablation)GLP on Llama1B · Diffusion lossloss power law: irreducible error 0.52Improved
prefilter top-512 candidate features before 1-D probing (ablation)Raw layer output / MLP neuron · 1-D probingprobe AUC: 0.82 -> 0.82No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy