AI Paper Atlas

Research map / Reinforcement learning / MDPs and planning

Is the Bellman residual a bad proxy?

Matthieu Geist, Bilal Piot, Olivier Pietquin

NeurIPS 2017 · Reinforcement learning › MDPs and planning · approach: Average-reward and regularized MDPs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Minimize the policy Bellman residual instead of maximizing mean value, using the distribution of interestGarnet G(30,4,2) with F(8,3), ν = μ · 100 randomly generated Garnetsnormalized value errorHurt
Maximize mean value instead of minimizing Bellman residual, using the distribution of interestGarnet G(30,4,2) with F(8,3) · 100 randomly generated Garnetsnormalized value errorImproved
Use the ideal occupancy distribution d_{μ,π*} rather than μ for optimizationPS · Garnet, value error measured under μnormalized value errorNo clear effect
Use the ideal occupancy distribution d_{μ,π*} rather than μ for optimizationRPS · Garnet, value error measured under μnormalized value errorNo clear effect
Increase sampling-distribution mismatch from the ideal occupancy distributionPS · Garnet with μ concentrated on one starting state; ν_α mixtureintegrated normalized value errorNo clear effect
Increase sampling-distribution mismatch from the ideal occupancy distributionRPS · Garnet with μ concentrated on one starting state; ν_α mixtureintegrated normalized value errorHurt
Optimize the Bellman residual rather than mean valueRPS versus PS · Garnet with varying sampling distributionsintegrated Bellman residualImproved
Use the ideal occupancy distribution for RPS optimization while evaluating residual under μRPS · Garnet with ν = d_{μ,π*}Bellman residual under μHurt
Decrease the Bellman residual through policy residual searchRPS(μ) · Garnet with ν = μnormalized value errorImproved
Increase mean value through policy searchPS(μ) · Garnet with ν = μBellman residualMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy