AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

State-Action Similarity-Based Representations for Off-Policy Evaluation

Brahma S Pavse, Josiah P. Hanna

NeurIPS 2023 · Reinforcement learning › Policy gradient and actor-critic · approach: Confidence intervals and OPE benchmarks · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
feed ROPE representations into FQE instead of raw state-actionsFQE · HalfCheetah-medium (D4RL)RMAE: 0.770 -> 0.247Improved
feed ROPE representations into FQE instead of raw state-actionsFQE · HalfCheetah-medium-expert (D4RL)RMAE: 0.374 -> 0.078Improved
feed ROPE representations into FQE instead of raw state-actionsFQE · HalfCheetah-random (D4RL)RMAE: 0.807 -> 0.990Hurt
use ROPE representations where FQE divergesFQE · Walker2D-medium (D4RL)RMAE: Diverged -> 0.462Improved
use ROPE representations where FQE divergesFQE · Hopper-medium (D4RL)RMAE: Diverged -> 0.208Improved
ROPE encoder instead of BCRL representationBCRL · Hopper-medium (D4RL)RMAE: 61.223 -> 0.208Improved
ROPE encoder instead of BCRL representationBCRL · Walker2D-medium-expert (D4RL)RMAE: 0.962 -> 0.252Improved
ROPE vs BCRL on random datasetBCRL · HalfCheetah-random (D4RL)RMAE: 0.979 -> 0.990No clear effect
ROPE vs state-action metrics (on-policy MICO) for groupingon-policy MICO · 3x3 gridworldgrouping aligned with q^pi_eImproved
vary ROPE encoder output dimension (ablation)ROPE · Walker2D-medium (D4RL)RMAENo clear effect
vary beta (angular distance weight) (ablation)ROPE · Walker2D-medium (D4RL)RMAENo clear effect
double FQE network capacity (FQE-deep) (ablation)FQE · performance profile over runsfraction of runs with RMAE <= 10No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy