Research map / Large language models / Retrieval and knowledge
BESPOKE: Benchmark for Search-Augmented Large Language Model Personalization via Diagnostic Feedback
Hyunseo Kim, Sangam Lee, Kwangwook Seo, Dongha Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add user-history profile (selected, query-aware) (ablation) | o3-search · BESPOKE | average score: 55.93 -> 62.48 | Improved |
| add user-history profile (selected, query-aware) (ablation) | GPT-4o-search · BESPOKE | average score: 47.56 -> 57.00 | Improved |
| make user context query-aware instead of static (ablation) | pplx-sonar · BESPOKE | average score: 49.76 -> 57.77 | Improved |
| select only query-relevant history sessions (ablation) | pplx-sonar · BESPOKE | average score: 57.77 -> 59.74 (with raw vs profile 49.79) | Improved |
| use structured profile instead of raw history (ablation) | pplx-sonar · BESPOKE | average score: 49.79 -> 59.74 | Improved |
| add human feedback to the evaluator context (ablation) | GPT-5 evaluator · meta-evaluation | Pearson correlation avg: 0.809 -> 0.847 | Improved |
| add personalization context to evaluator (ablation) | GPT-5 evaluator · meta-evaluation | Pearson correlation avg: 0.470 -> 0.809 | Improved |
| expand query with CoT before history retrieval | Stella-V5-1.5B · history retrieval | nDCG@10: 0.0820 -> 0.3809 | Improved |
| expand query with pseudo-history | Stella-V5-1.5B · history retrieval | nDCG@10: 0.0820 -> 0.3807 | Improved |
| pseudo-history instead of CoT expansion | Stella-V5-1.5B · history retrieval | nDCG@10: 0.3809 vs 0.3807 | No clear effect |
| provide gold information instead of web search results (ablation) | search-augmented LLMs · BESPOKE | need alignment, content depth | Improved |
| increase search noise to 70% (ablation) | pplx-sonar-reasoning · BESPOKE noise study | Avg.: 61.40 -> 49.87 | Hurt |
Closest papers by meaning
- Rethinking Personalization in Large Language Models at the Token Level (ICML 2026 · similarity 0.88)
- HYDRA: Model Factorization Framework for Black-Box LLM Personalization (NeurIPS 2024 · similarity 0.87)
- RAP: Retrieval-Augmented Personalization for Multimodal Large Language Models (CVPR 2025 · similarity 0.87)
- Do LLMs Recognize Your Preferences? Evaluating Personalized Preference Following in LLMs (ICLR 2025 · similarity 0.86)
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.86)
- Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
