Research map / Large language models / Language, safety and interpretability
Leveraging semantic similarity for experimentation with AI-generated treatments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes double kernel representation learning, factorizing treatment effects into inner products of kernel-based treatment-embedding and user-covariate representations, with an alternating-minimization estimator and explore-then-commit bandit for LLM-generated treatments, evaluated on semi-synthetic Upworthy and MIND data.
Problem
Estimating heterogeneous treatment effects and running adaptive experiments over a large, high-dimensional space of LLM-generated treatment variants, where discrete A/B testing is underpowered and ignores semantic similarity among variants.
Approach
Factorize the conditional treatment effect as an inner product of low-dimensional kernel representations of treatment embeddings and user covariates (double RKHS low-rank model), fit by alternating ridge-type minimization, and use the learned model in an explore-then-commit adaptive design.
Key results
- On the Upworthy semi-synthetic setup at rank 2, DKRL achieves test error 0.0036 versus 0.0104 for SIN and 0.0080 for ProdKernel.
- Explore-then-commit with covariates yields sublinear cumulative regret, whereas ignoring user covariates leads to linear regret.
- Theory gives an estimation error bound of order lambda_n^2 r for the signal matrix and a regret bound of order T^{2/3}.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DKRL instead of ProdKernel | Upworthy synthetic data, rank 2 · Upworthy synthetic, rank 2 | test error: 0.0080 (0.0013) -> 0.0036 (0.0010) | Helped |
| Use DKRL instead of SIN | Upworthy synthetic data, rank 2 · Upworthy synthetic, rank 2 | test error: 0.0104 (0.0017) -> 0.0036 (0.0010) | Helped |
| Use DKRL instead of ProdKernel | Upworthy synthetic data, rank 3 · Upworthy synthetic, rank 3 | test error: 0.0090 (0.0013) -> 0.0048 (0.0013) | Helped |
| Use DKRL instead of SIN | Upworthy synthetic data, rank 3 · Upworthy synthetic, rank 3 | test error: 0.0113 (0.0016) -> 0.0048 (0.0013) | Helped |
| Use DKRL instead of ProdKernel | Upworthy synthetic data, rank 5 · Upworthy synthetic, rank 5 | test error: 0.0115 (0.0014) -> 0.0078 (0.0012) | Helped |
| Use DKRL instead of SIN | Upworthy synthetic data, rank 5 · Upworthy synthetic, rank 5 | test error: 0.0144 (0.0018) -> 0.0078 (0.0012) | Helped |
| Use DKRL instead of ProdKernel | Upworthy synthetic data, rank 7 · Upworthy synthetic, rank 7 | test error: 0.0136 (0.0017) -> 0.0108 (0.0015) | Helped |
| Use DKRL instead of SIN | Upworthy synthetic data, rank 7 · Upworthy synthetic, rank 7 | test error: 0.0172 (0.0021) -> 0.0108 (0.0015) | Helped |
| Include user-level covariates and treatment-user interactions instead of using only treatment representations | Explore-then-commit online experimentation · Upworthy bandit simulation | cumulative regret | Helped |
Surprising findings
- Neural network approaches are inferior here, plausibly because they fail to learn the low-rank representation.
- Ignoring user-level covariates in the bandit causes failed exploration and linear regret.
Limitations
- Theory is limited to a fixed-basis setting; infinite-dimensional bases are left for future work.
- Experiments are semi-synthetic, with simulated covariates and synthetic outcomes, rather than real online experiments.
- Upworthy user covariates are simulated Gaussian vectors because they are unavailable in the dataset.
Experimental setup
Datasets: Upworthy Research Archive MIND ASOS E-Commerce Dataset
Benchmarks: Upworthy Research Archive (semi-synthetic) MIND (semi-synthetic)
Metrics: train error test error RMSE cumulative regret computation time
Baselines: SIN ProdKernel LASSO XGBoost feed-forward neural network kernel regression explore-then-commit with treatment representations only
Builds on: Structured Intervention Network kernelized probabilistic matrix factorization R-learner explore-then-commit
Closest papers by meaning
- Representation-Based Exploration for Language Models: From Test-Time to Post-Training (ICLR 2026 · similarity 0.84)
- Optimal Representation Size: High-Dimensional Analysis of Pretraining and Linear Probing (NeurIPS 2026 · similarity 0.83)
- LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs (ICML 2026 · similarity 0.83)
- Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation (ICML 2024 · similarity 0.83)
- From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.83)
- Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.