AI Paper Atlas

Research map / Large language models / Alignment and preferences

RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards

Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Ellie Evans, Daniel Egert, Hoo-Chang Shin, Felipe Soares, Yi Dong, Oleksii Kuchaiev

ICLR 2026 · Large language models › Alignment and preferences · approach: Reward modeling for diverse preferences · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with principle-conditioned Yes/No (Flexible Principles) instead of Bradley-TerryLlama-3.3-70B scalar RM · RM-Benchoverall accuracy: 78.5 -> 83.6Improved
train with principle-conditioned Yes/No (Flexible Principles) instead of Bradley-TerryLlama-3.3-70B scalar RM · JudgeBenchoverall accuracy: 68.9 -> 76.3Improved
train with principle-conditioned Yes/No instead of Bradley-TerryLlama-3.1-8B scalar RM · JudgeBench (8B base)overall accuracy: 62.9 -> 64.0Improved
replace scalar RM with reasoning GenRM trained with GRPOFlexible Principles RM · JudgeBenchoverall accuracy: 76.3 -> 81.4Improved
replace scalar RM with reasoning GenRMFlexible Principles RM · PrincipleBenchoverall micro-average: 91.6 -> 83.8Hurt
raise consensus similarity threshold from 0.8 to 0.9 (ablation)Flexible Principles ScalarRM · JudgeBenchoverall accuracy: 76.3 -> 73.7Hurt
lower consensus similarity threshold from 0.8 to 0.7 (ablation)Flexible Principles ScalarRM · JudgeBenchoverall accuracy: 76.3 -> 72.3Hurt
train on multiple flexible principles instead of one fixed principle (test-time fixed) (ablation)Flexible Principles ScalarRM · RM-Benchoverall accuracy: 79.9 -> 81.9Improved
allow flexible principle selection at test time instead of fixing accuracy of information (ablation)Flexible Principles ScalarRM · JudgeBenchoverall accuracy: 70.9 -> 76.3Improved
remove equal Yes/No weighting (keep natural imbalance) (ablation)Flexible Principles ScalarRM · RM-Benchoverall accuracy: 81.5 -> 83.6No clear effect
train Qwen3-32B with RLBFF GenRM reward instead of Bradley-Terry RMQwen3-32B · Arena Hard v2score: 47.5 -> 55.6Improved
train Qwen3-32B with RLBFF GenRM reward instead of Bradley-Terry RMQwen3-32B · WildBench Overallscore: 67.38 -> 70.33Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy