Research map / Reinforcement learning / RL from feedback

Testing and Estimation of Contextual Generalized Thurstone Models

NeurIPS 2026 · Reinforcement learning › RL from feedback · approach: Theory of RLHF

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies testing and estimation for contextual generalized Thurstone models of pairwise preference data, motivated by the unclear theoretical justification of BTL-style models used in RLHF.

Problem

Preference learning models such as Bradley-Terry-Luce (BTL) are used to learn latent item scores from pairwise comparisons (e.g., reward models in RLHF), but their theoretical justification is often unclear.

Approach

Analyzes contextual generalized Thurstone models as a broader family of pairwise comparison models, providing methods to test model fit and estimate latent scores; the abstract is truncated so the specific construction is not stated.

Experimental setup

Builds on: Bradley-Terry-Luce model Thurstone model

Closest papers by meaning

• Hypothesis Testing for Generalized Thurstone Models (ICML 2025 · similarity 0.89)
• Iterative Preference Learning from Human Feedback: Bridging Theory and Practice for RLHF under KL-constraint (ICML 2024 · similarity 0.85)
• Bayesian Preference Learning for Test-Time Steerable Reward Models (NeurIPS 2026 · similarity 0.85)
• Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.85)
• Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.85)
• Provably Efficient Online RLHF with One-Pass Reward Modeling (NeurIPS 2025 · similarity 0.85)