Research map / Large language models / Alignment and preferences
P-GenRM: Personalized Generative Reward Model with Test-time User-based Scaling
Pinyi Zhang, Ting-En Lin, Yuchuan Wu, Jingyang Chen, Zongqi Wang, Hua Yang, Xu Ze, Fei Huang, Yongbin Li, Kai Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add prototype-level scaling to individual-level scaling (Ind-16, Pro-8 vs Ind-32) (ablation) | P-GenRM 8B · Chatbot Arena-Personalized | accuracy (%): 75.59 -> 75.92 | Improved |
| add test-time user-based scaling (Ind-16, Pro-8) | P-GenRM 8B · PRISM-Personalized | accuracy (%): 65.32 -> 68.06 | Improved |
| use prototype-level scaling alone (Ind-0, Pro-8) (ablation) | P-GenRM 8B · Chatbot Arena-Personalized | accuracy (%): 72.68 -> 66.90 | Hurt |
| increase prototype-level samples from 8 to 16 with Ind-16 (ablation) | P-GenRM 8B · Chatbot Arena-Personalized | accuracy (%): 75.92 -> 72.59 | Hurt |
| add curriculum learning (ablation) | P-GenRM 8B · Chatbot Arena-Personalized | accuracy (%): 71.07 -> 72.68 | Improved |
| add process reward in RL (ablation) | P-GenRM 8B · Chatbot Arena-Personalized | accuracy (%): 70.22 -> 71.07 | Improved |
| add outcome reward in RL (ablation) | P-GenRM 8B · PRISM-Personalized | accuracy (%): 60.94 -> 63.82 | Improved |
| add RL stage after SFT (ablation) | P-GenRM 8B · PRISM-Personalized | accuracy (%): 57.08 -> 63.82 | Improved |
| add SFT (persona-guided scoring induction) stage (ablation) | P-GenRM 8B · Chatbot Arena-Personalized | accuracy (%): 56.37 -> 66.76 | Improved |
| replace static SynthesizeMe persona with adaptive PSI (ablation) | o3 judge · PRISM-Personalized | accuracy (%): 58.49 -> 63.87 | Improved |
| replace SynthesizeMe with PSI (ablation) | Qwen3-8B judge · Chatbot Arena-Personalized | accuracy (%): 62.57 -> 64.22 | Improved |
| increase prototypes from 0 to 50 (Ind-8, Pro-4) (ablation) | P-GenRM 8B · PRISM-Personalized | accuracy (%): 65.32 -> 67.54 | Improved |
Closest papers by meaning
- Capturing Individual Human Preferences with Reward Features (NeurIPS 2025 · similarity 0.91)
- GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-Time Alignment (ICLR 2025 · similarity 0.91)
- GRAM: A Generative Foundation Reward Model for Reward Generalization (ICML 2025 · similarity 0.90)
- Test-Time Personalization: A Diagnostic Framework and Probabilistic Fix for Scaling Failures (NeurIPS 2026 · similarity 0.90)
- Rethinking Personalized Generation: Test-time Alignment via Factorized Ranking Models (NeurIPS 2026 · similarity 0.90)
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
