Research map / Large language models / Reasoning and chain of thought

Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Test-Time Personalization (scaling best-of-N sampling from a personalized policy with a personalized reward model), diagnoses reward-model failures via a unified scaling law, and introduces a probabilistic (Gaussian NLL) user reward model that mitigates collapse and reward hacking.

Problem

Best-of-N selection with learned personalized reward models fails to scale for LLM personalization: global and per-user reward models perform near random, due to user-level collapse and query-level reward hacking.

Approach

Derive an oracle log-N scaling law and a unified scaling law that predicts Best-of-N curves from reward-model correlation statistics, then use a probabilistic reward model with a learned variance head (trained with Gaussian NLL plus margin loss, inference via mean only) to remove collapse and reduce hacking.

Key results

• Probabilistic User RM is the best non-oracle selector on the RAG policy across all five tasks, closing 51% to 84% of the oracle gap at N=30.
• Scaling law predicts observed Best-of-N curves at mean relMAE 1.7% for Prob RM versus 4.4% for Global RM and 2.7% for Det RM.
• Collapse rate drops from 0.15 (deterministic) to 0 with probabilistic RM, and hacking rate from 0.25 to 0.06 averaged across tasks.

Surprising findings

• A GPT-class LLM-as-judge achieves only r=0.099 correlation with ROUGE on LaMP-4, essentially random for stylistic personalization.
• Global RM performs no better than random selection on LaMP-5 and falls below random on LaMP-4 under Persona+RAG due to reward hacking.

Limitations

• Evaluation limited to personalized text generation with ROUGE-derived ground-truth rewards; generalization to dialogue or agentic tasks untested.
• Assumes sufficient per-user data to train a reward model; cold-start users are out of scope.
• Requires training separate per-user reward models, which may be costly at scale.

Experimental setup

Datasets: LaMP-4 LaMP-5 LongLaMP Abstract LongLaMP Product Review LongLaMP Topic Writing

Benchmarks: LaMP-4 LaMP-5 LongLaMP Abstract LongLaMP Product Review LongLaMP Topic Writing

Metrics: ROUGE-1 ROUGE-L BERTScore relMAE

Baselines: Random selection Global RM Deterministic User RM Per-user LoRA personalization Oracle selection

Builds on: LaMP LongLaMP retrieval-augmented generation Gaussian negative log-likelihood training

Closest papers by meaning

• Rethinking Personalized Generation: Test-time Alignment via Factorized Ranking Models (NeurIPS 2026 · similarity 0.89)
• On the Limits of Test-Time Compute: Sequential Reward Filtering for Better Inference (ICML 2026 · similarity 0.87)
• Rethinking Personalization in Large Language Models at the Token Level (ICML 2026 · similarity 0.87)
• Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.87)
• Optimal Aggregation of LLM and PRM Signals for Efficient Test-Time Scaling (ICLR 2026 · similarity 0.86)
• Scaling Test-Time Compute Without Verification or RL is Suboptimal (ICML 2025 · similarity 0.86)