Research map / Large language models / Alignment and preferences
LLM-as-judge and feedback data
Collect scaled AI feedback and study human-versus-LLM judgment in evaluation.
111 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2018–2026) · an approach in Alignment and preferences, keyword cluster “feedback · human · rlhf”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
25 papers in 2023–24 and 80 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 19.1% to 15.3%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2018 · Exploring Implicit Feedback for Open Domain Conversation Generation (AAAI 2018 · 21 citations)
- 2020 · Balancing Quality and Human Involvement: An Effective Approach to Interactive Neural Machine Translation (AAAI 2020 · 19 citations)
- 2022 · Training language models to follow instructions with human feedback (NeurIPS 2022 · 1,219 citations)
- 2023 · Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (NeurIPS 2023 · 810 citations)
- 2024 · Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference (ICML 2024 · 103 citations)
- 2025 · Evaluating the Evaluator: Measuring LLMs’ Adherence to Task Evaluation Instructions (AAAI 2025 · 16 citations)
- 2026 · Preference Leakage: A Contamination Problem in LLM-as-a-judge (ICLR 2026 · 2 citations)
Earliest papers here
- Exploring Implicit Feedback for Open Domain Conversation Generation (AAAI 2018 · 21 citations)
- Balancing Quality and Human Involvement: An Effective Approach to Interactive Neural Machine Translation (AAAI 2020 · 19 citations)
- Human-Machine Collaboration for Fast Land Cover Mapping (AAAI 2020 · 18 citations)
What has been tried
57 papers here report 626 measured changes (a component added, removed or swapped, compared with a baseline): helped 59%, no clear effect 10%, hurt 24%, mixed 7%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Fine-tune GPT-3.5 on 10 self-recognition examples from XSUM | XSUM · self-preference | 0.582 -> 0.657 | Helped | LLM Evaluators Recognize and Favor Their Own Generations (NeurIPS 2024 · 65 citations) |
| train UltraRM with fine-grained UltraFeedback scores instead of overall scores | OpenAI WebGPT · accuracy | 62.0 -> 65.2 | Helped | ULTRAFEEDBACK: Boosting Language Models with Scaled AI Feedback (ICML 2024 · 22 citations) |
| Randomize answer-choice ordering | ACS · response entropy | approximately~1 | Helped | Questioning the Survey Responses of Large Language Models (NeurIPS 2024 · 22 citations) |
| Use full rubric instead of a criteria-specific prompt | All datasets, all criteria · Pearson correlation | 0.367 -> 0.334 | Hurt | Evaluating the Evaluator: Measuring LLMs’ Adherence to Task Evaluation Instructions (AAAI 2025 · 16 citations) |
| Remove the system role prompt | Crimes, English, SelfCipher · unsafety rate | 70.9 -> 1.0 | Hurt | GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher (ICLR 2024 · 14 citations) |
| add synthetic safety SFT data but omit RBR reward | Internal safety, medium policy model · F1-Score | 94.1 ± 0.8% -> 92.3 ± 0.9% | Hurt | Rule Based Rewards for Language Model Safety (NeurIPS 2024 · 8 citations) |
| Use a high-assertiveness preamble | Confounder experiment · factuality error detection relative to expert annotation | -16.1 -> -22.3 | Hurt | Human Feedback is not Gold Standard (ICLR 2024 · 7 citations) |
| Add style control to model scoring | Style-controlled Chatbot Arena Hard Prompt models · Spearman Correlation | 94.9% -> 98.6% | Helped | From Crowdsourced Data to High-quality Benchmarks: Arena-Hard and Benchbuilder Pipeline (ICML 2025 · 6 citations) |
| Add Gradient Cuff to Self-Reminder | Malicious jailbreak queries, average across 6 attacks · TPR | 0.820±0.016 -> 0.920±0.011 | Helped | Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes (NeurIPS 2024 · 5 citations) |
| add task-specific checklists to the WB-Score prompt | Chatbot Arena Elo correlation, all models · Pearson correlation | 0.905 -> 0.925 | Helped | WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild (ICLR 2025 · 4 citations) |
| Train with identical initial states for gameable and non-gameable users | Therapy-talk, same initial states · harmful trajectories | ~75% vs ~20% | Mixed | On Targeted Manipulation and Deception when Optimizing LLMs for User Feedback (ICLR 2025 · 4 citations) |
| use the Arena-Hard judge to verify candidate solutions rather than solve the problem directly | JudgeBench Overall · accuracy | 54.57 to 56.57 | Helped | JudgeBench: A Benchmark for Evaluating LLM-Based Judges (ICLR 2025 · 3 citations) |
What reviewers will expect
Datasets and baselines named in the 44 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Chatbot Arena · 4 of 44 papers
- MT-Bench · 4 of 44 papers
- Flask · 2 of 44 papers
- WildChat · 2 of 44 papers
- HelpSteer · 2 of 44 papers
- BIGGen Bench · 2 of 44 papers
- UltraFeedback · 2 of 44 papers
- RewardBench · 2 of 44 papers
Baselines
- SFT · 3 of 44 papers
- Prometheus-2 · 2 of 44 papers
- MT-Bench · 2 of 44 papers
- zero-shot · 2 of 44 papers
- Random Selection · 2 of 44 papers
- PandaLM · 2 of 44 papers
- Reminder · 2 of 44 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 105 papers; from OpenAlex and the papers' affiliations).
- University of California, Berkeley · 9 papers
- Google · 6 papers
- Stanford University · 5 papers
- New York University · 5 papers
- Microsoft · 4 papers
- Meta · 4 papers
- Tsinghua University · 4 papers
- Carnegie Mellon University · 4 papers
- University College London · 3 papers
- Cohere · 3 papers
Use it
Download BibTeX of 28 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Steering and correcting LLM behavior (127 papers)
- Reward modeling for diverse preferences (90 papers)
- Reward model design and preference models (88 papers)
Elsewhere in the topic
- Robust and calibrated DPO (118 papers)
- DPO design choices (73 papers)
- Active and token-level preference learning (58 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).