Research map / Large language models / Alignment and preferences
ELSPR: Evaluator LLM Training Data Self-Purification on Non-Transitive Preferences via Tournament Graph Reconstruction
Yan Yu, Yilun Liu, Minggui He, Shimin Tao, Weibin Meng, Xinhua Yang, Li Zhang, Hongxia Ma, Dengye Li, Daimeng Wei, Boxing Chen, Fuliang Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune on cleaned (tournament-filtered) data instead of raw | Qwen2.5-7B-Instruct evaluator · cross-dataset average | preference non-transitivity: 64.3% -> 50.5% | Improved |
| fine-tune on cleaned data instead of raw | Qwen2.5-7B-Instruct evaluator · cross-dataset average | normalized structural entropy: 0.811 -> 0.723 | Improved |
| fine-tune on cleaned data instead of raw | Qwen2.5-7B-Instruct evaluator · Helpful_Base | preference non-transitivity: 62.0% -> 44.9% | Improved |
| fine-tune on cleaned data instead of raw | LLaMA3.1-8B-Instruct evaluator · Helpful_Base | preference non-transitivity: 59.0% -> 40.2% | Improved |
| fine-tune on targeted-filtered data instead of random 20% filtering (ablation) | Qwen2.5-7B-Instruct evaluator · Helpful_Base | preference non-transitivity: 60.6% -> 44.9% | Improved |
| fine-tune on random 20% filtered data instead of raw (ablation) | Qwen2.5-7B-Instruct evaluator · Helpful_Base | preference non-transitivity: 62.0% -> 60.6% | No clear effect |
| fine-tune on cleaned data instead of raw | Qwen2.5-7B-Instruct evaluator · Vicuna (tie-allowed prompt) | preference non-transitivity | Mixed |
| fine-tune on cleaned data instead of raw | Qwen2.5-7B-Instruct evaluator · Vicuna (tie-allowed prompt) | tie probability: 10.06% -> 6.13% | Improved |
| keep cleaned instead of discarded preference pairs | preference data · 5 datasets | human consistency: 34.4% -> 52.6% | Improved |
| keep cleaned instead of discarded preference pairs | preference data · 5 datasets | model-human agreement: 51.2% -> 80.6% | Improved |
| fine-tune on cleaned data instead of raw | evaluator in MT-bench framework · Helpful_Base | std of adjusted win rates: 12.2% -> 16.1% | Improved |
| fine-tune on cleaned data instead of raw | evaluator in MT-bench framework · Self-instruct | std of adjusted win rates: 8.2% -> 9.7% | Improved |
Closest papers by meaning
- Investigating Non-Transitivity in LLM-as-a-Judge (ICML 2025 · similarity 0.89)
- Reward-Augmented Data Enhances Direct Preference Alignment of LLMs (ICML 2025 · similarity 0.89)
- Bridging Human and LLM Judgments: Understanding and Narrowing the Gap (NeurIPS 2025 · similarity 0.89)
- 3D-Properties: Identifying Challenges in DPO and Charting a Path Forward (ICLR 2025 · similarity 0.89)
- Improving Context-Aware Preference Modeling for Language Models (NeurIPS 2024 · similarity 0.89)
- Less is More: Improving LLM Alignment via Preference Data Selection (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
