Research map / Reinforcement learning / Policy gradient and actor-critic

POTEC: Off-Policy Contextual Bandits for Large Action Spaces via Policy Decomposition

ICLR 2025 · Reinforcement learning › Policy gradient and actor-critic · approach: Importance-weighted OPE estimators

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes POTEC, a two-stage off-policy contextual bandit algorithm that decomposes the policy for large discrete action spaces to reduce bias and variance.

Problem

Off-policy learning of contextual bandit policies in large discrete action spaces, where reward-regression and importance-weighted policy gradient methods suffer from excessive bias or variance.

Approach

A two-stage policy decomposition: a clustering-based first stage narrows the action space and a second stage selects within clusters, allowing lower-variance, lower-bias policy gradient estimation than standard importance-weighted methods.

Closest papers by meaning

• Off-Policy Learning in Large Action Spaces: Optimization Matters More Than Estimation (ICML 2026 · similarity 0.92)
• Off-Policy Evaluation for Large Action Spaces via Conjunct Effect Modeling (ICML 2023 · similarity 0.92)
• Contextual Bandits with Large Action Spaces: Made Practical (ICML 2022 · similarity 0.90)
• A General Framework for Off-Policy Learning with Partially-Observed Reward (ICLR 2025 · similarity 0.87)
• Kernel Metric Learning for In-Sample Off-Policy Evaluation of Deterministic RL Policies (ICLR 2024 · similarity 0.87)
• Logarithmic Smoothing for Pessimistic Off-Policy Evaluation, Selection and Learning (NeurIPS 2024 · similarity 0.87)