Research map / Large language models / Alignment and preferences
T-POP: Test-Time Personalization with Online Preference Feedback
Zikun Qu, Min Zhang, Mingze Kong, Xiang Li, Zhiwei Shang, Zhiyong Wang, Yikun Ban, Shuang Qiu, Yao Shu, Zhongxiang Dai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add gradient-norm uncertainty bonus instead of random exploration (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.51 -> 0.64 | Improved |
| use entropy bonus instead of random exploration (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.51 -> 0.53 | No clear effect |
| use Boltzmann exploration instead of random exploration (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.51 -> 0.57 | Improved |
| raise reward weight omega from 0.1 to 1.0 (ablation) | T-POP · three backbones | ArmoRM score | Improved |
| raise reward weight omega from 1.0 to 2.0 and 5.0 (ablation) | T-POP · three backbones | ArmoRM score | Hurt |
| increase candidate size k from 10 to 20 (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.642 -> 0.659 | Improved |
| increase candidate size k from 40 to 80 (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.656 -> 0.633 | Hurt |
| flip 30% of preference labels (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.656 -> 0.649 | No clear effect |
| flip 90% of preference labels (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.656 -> 0.493 | Hurt |
| inject 60% local noise in middle phase versus uniform noise (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.651 -> 0.643 | Hurt |
| inject 60% local noise in late phase versus uniform noise (ablation) | T-POP · Personal concise, Llama-3.1-8B | ArmoRM score: 0.651 -> 0.656 | No clear effect |
| apply T-POP to Llama-3.2-1B-Instruct | base model · ArmoRM personalization | ArmoRM score: 0.28 -> 0.44 | Improved |
Closest papers by meaning
- PersonalLLM: Tailoring LLMs to Individual Preferences (ICLR 2025 · similarity 0.90)
- Rethinking Personalized Generation: Test-time Alignment via Factorized Ranking Models (NeurIPS 2026 · similarity 0.89)
- Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures (NeurIPS 2026 · similarity 0.88)
- Rethinking Personalization in Large Language Models at the Token Level (ICML 2026 · similarity 0.88)
- PAD: Personalized Alignment of LLMs at Decoding-time (ICLR 2025 · similarity 0.88)
- Amulet: ReAlignment During Test Time for Personalized Preference Adaptation of LLMs (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
