Research map / Large language models / Alignment and preferences
Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PrefEval, a 3,000-pair long-context benchmark testing whether LLMs infer, remember, and follow user preferences across multi-session conversations, and shows that zero-shot accuracy collapses while reminder/RAG prompting and SFT help.
Problem
LLM chatbots struggle to proactively personalize responses to user preferences stated earlier in long, multi-session conversations, and no benchmark measured this preference-following ability (explicit and implicit) at scale.
Approach
A benchmark of 3,000 preference-query pairs across 20 topics and three preference forms (explicit, implicit choice-based, implicit persona-driven), interleaved with LMSYS-Chat-1M distractor turns up to 100k tokens, scored by both an LLM-judge generation task (four error types) and an MCQ classification task that correlates with it (r=0.73).
Key results
- Zero-shot preference-following accuracy falls below 10% at 10 turns (~3k tokens) for most evaluated models, e.g., Claude 3.5 Sonnet 0.07 and Gemini-1.5-Pro 0.07 at 10 turns.
- Reminder prompting raises accuracy substantially, e.g., Claude 3 Sonnet averages 99.03% with Reminder vs 13.41% zero-shot on 10-turn explicit preferences across 20 topics.
- Classification MCQ accuracy correlates with generation accuracy at r=0.73; SFT on Mistral-7B lifts zero-shot accuracy above the RAG baseline and generalizes to 70-turn contexts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a preference reminder | Proprietary LLMs · Travel restaurant, 10 turns | preference-following accuracy: 0.07 -> 0.45; 0.07 -> 0.91; 0.50 -> 0.98; 0.07 -> 0.98 | Helped |
| Add a preference reminder | Proprietary LLMs · Travel restaurant, 300 turns | preference-following accuracy: 0.02 -> 0.02; 0.09 -> 0.05; 0.14 -> 0.98; 0.05 -> 0.23 | Mixed |
| Increase conversation length | Zero-shot LLMs · Multi-session preference following | preference-following accuracy | Hurt |
| Use RAG instead of zero-shot prompting | Most evaluated LLMs · Explicit preference generation | preference-following accuracy | Helped |
| Use Reminder instead of more complex prompting | Evaluated LLMs · Explicit preference generation | preference-following accuracy | Helped |
| Use Self-Critic instead of zero-shot | Claude models and Mistral-8x7B · Classification task | classification accuracy | Hurt |
| Use implicit rather than explicit preferences | Evaluated LLMs · Preference following across preference forms | preference-following accuracy | Hurt |
| Add multiple non-conflicting preferences throughout the conversation | Claude 3 Sonnet + Reminder · Follow the initial preference | adherence accuracy | Helped |
| Add conflicting preferences, with the target preference stated later | Evaluated LLMs · Follow the later original preference | adherence accuracy | Helped |
| Fine-tune Mistral-7B on PrefEval | Mistral-7B · Unseen topics, generation task | preference-following accuracy | Helped |
| Train with 10 contextual turns rather than fewer contextual turns (ablation) | Fine-tuned Mistral-7B · 70-turn contexts | preference-following accuracy | Helped |
| Use RAG with top 10 rather than top 5 retrieved exchanges (ablation) | RAG · Explicit preference following | preference-following performance | No clear effect |
Surprising findings
- Multiple stated preferences, including conflicting ones, improved adherence to the original preference rather than hurting it.
- Reminder prompting outperformed more complex methods such as Self-Critic and few-shot CoT.
Limitations
- Preferences and queries are synthetic, LLM-assisted and manually filtered rather than real user preferences.
- Benchmark measures preference following rather than factual accuracy of recommendations.
- Implicit choice-based preferences may be misinferred since only one option aligns with the preference, with an estimated residual error.
Experimental setup
Datasets: PrefEval LMSYS-Chat-1M
Benchmarks: PrefEval
Metrics: preference-following accuracy MCQ classification accuracy error type rate
Baselines: Zero-shot Reminder Self-Critic Few-shot CoT RAG (top-5)
Builds on: LMSYS-Chat-1M intrinsic self-correction chain-of-thought
Closest papers by meaning
- PersonalLLM: Tailoring LLMs to Individual Preferences (ICLR 2025 · similarity 0.92)
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.90)
- Aligning LLMs by Predicting Preferences from User Writing Samples (ICML 2025 · similarity 0.89)
- Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions (ICLR 2025 · similarity 0.89)
- PersonaVLM: Long-Term Personalized Multimodal LLMs (CVPR 2026 · similarity 0.89)
- Aligning to Thousands of Preferences via System Message Generalization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.