Research map / Large language models / Alignment and preferences
References Improve LLM Alignment in Non-Verifiable Domains
Kejian Shi, Yixin Liu, PeiFeng Wang, Alexander Fabbri, Shafiq Joty, Arman Cohan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reference output with explicit usage instructions (RefEval vs reference-free LLMBar-Base) | LLM judge · 5 pairwise datasets (11 judges) | accuracy (%): 72.3 -> 79.1 | Improved |
| add references with RefEval prompt over reference-free Ref-Free prompt | LLM judge · 5 pairwise datasets (11 judges) | accuracy (%): 73.7 -> 79.1 | Improved |
| naive reference inclusion (LLMBar-Ref) over LLMBar-Base | LLM judge · 5 pairwise datasets (11 judges) | accuracy (%): 72.3 -> 74.0 | Improved |
| add RefEval to small judge llama-3-8b | llama-3-8b judge · 5 pairwise datasets | accuracy (%): 60.1 -> 77.5 | Improved |
| add RefEval to strong judge qwen-2.5-72b | qwen-2.5-72b judge · 5 pairwise datasets | accuracy (%): 79.4 -> 84.6 | Improved |
| use human-edited Oracle reference with RefEval (GPT-4o judge) | GPT-4o judge · LLMBar-Adversarial | accuracy (%): 86.8 -> 88.4 | Improved |
| use weaker references (Mistral-Nemo-12B) with RefEval vs reference-free (ablation) | Llama-3.1-70B judge · 4 pairwise datasets | accuracy (%): 80.3 -> 85.0 | Improved |
| add multi-reference voting to RefEval | RefEval · 4 datasets (11 judges) | accuracy: 0.782 -> 0.789 | Improved |
| add DPO stage with reference-free self-judge after distillation | Llama-3-8B-Instruct · AlpacaEval LC | win rate (%): 53.9 -> 67.5 | Improved |
| use reference-guided self-judge instead of reference-free self-judge in DPO | Llama-3-8B-Instruct · AlpacaEval LC | win rate (%): 67.5 -> 73.1 | Improved |
| use reference-guided self-judge instead of reference-free self-judge in DPO | Qwen2.5-7B-SFT · Arena-Hard | score (%): 71.8 -> 74.1 | Improved |
| use RefEval judge instead of ROUGE reward in DPO | Llama-3-8B-Instruct · AlpacaEval LC | win rate (%): 56.4 -> 73.1 | Improved |
Closest papers by meaning
- From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation (ICLR 2026 · similarity 0.89)
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.89)
- LaSeR: Reinforcement Learning with Last-Token Self-Rewarding (ICLR 2026 · similarity 0.88)
- The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning (NeurIPS 2025 · similarity 0.88)
- The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning (ICLR 2024 · similarity 0.88)
- BLEUBERI: BLEU is a surprisingly effective reward for instruction following (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
