AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Uncertainty-Guided Reward Labeling for Reinforcement Learning under Limited Feedback

Renhao Zhang, Shreyas Chaudhari, Bruno Silva

NeurIPS 2026 · Reinforcement learning › Offline RL · approach: Diffusion models for offline decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes an uncertainty-guided method for choosing which offline samples to reward-label under limited feedback in RL.

Problem

In reinforcement learning from limited feedback (RLLF), only a small fraction of an offline dataset can be reward-labeled, so the question is which samples to label to learn a strong policy. Prior work treats downstream policy learning as a black box.

Approach

Selects reward-labeling samples using uncertainty guidance, coupling the selection stage with downstream policy learning rather than treating policy learning as a black box (based on abstract only; details beyond the abstract are not stated).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
multiply reachability by uncertainty-reduction instead of either alone (ablation)SURE · maze2d-largepercentage feedback to reach 95%: 2.51% (Reach) -> 0.15%Improved
multiply reachability by uncertainty-reduction instead of uncertainty alone (ablation)SURE · halfcheetahpercentage feedback to reach 95%: 2.49% -> 0.71%Improved
replace exact one-step gain with computable SURE score (ablation)SURE · exact linear MDPlabels to reach 95%: 12 -> 12No clear effect
use SURE instead of uniform label selectionSURE · exact linear MDP (48 states)labels to reach 95%: 132 -> 12Improved
use SURE instead of uniform selection with CQL backboneSURE · HalfCheetahpercentage feedback to reach 90%: 2.44% -> 0.55%Improved
use SURE instead of uniform selection with TD3+BC backboneSURE · Maze2d-Mediumpercentage feedback to reach 90%: 1.92% -> 0.14%Improved
use multi-step successor reachability instead of one-step predecessor guidanceReach vs RLLF-Guided · maze2d-mediumpercentage feedback to reach 90%: 1.45% -> 0.50%Improved
use SURE instead of IDRLSURE · halfcheetahpercentage feedback to reach 95%: 6.18% -> 0.71%Improved
use SURE instead of Reach at the 80% levelSURE · maze2d-mediumpercentage feedback to reach 80%: 0.06% (Reach) vs 0.05%No clear effect
use SURE instead of Uncertainty at 80% levelSURE · halfcheetahpercentage feedback to reach 80%: 0.31% (Uncertainty) vs 0.30%No clear effect

Experimental setup

Builds on: reward selection under limited feedback

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy