Research map / Optimization / Nonconvex and smooth optimization
Statistical Complexity of Soft Bellman Residual Minimization
Enoch H. Kang, Kyoungseok Jang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical analysis of the statistical complexity of soft (entropy-regularized) Bellman residual minimization for offline RL with neural networks trained by stochastic gradient descent.
Problem
The statistical complexity of gradient-based soft Bellman residual minimization with neural networks under stochastic gradient descent is largely unknown, despite recent global convergence results for the optimization side in offline RL.
Approach
Provides a statistical complexity analysis for soft BRM trained with SGD in offline RL, extending prior globally convergent gradient-based soft BRM results to the sample-complexity setting. Only the abstract was available, so the specific technical contribution beyond this is not stated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| analyze soft BRM under SGDA with PL/QG geometry rather than convex-concave stability | soft BRM with SGDA · toy soft MDP, stability vs n | slope of primal argument stability: slopes -1.08 to -1.05 | Improved |
| use a single dependent trajectory instead of i.i.d. data (ablation) | SGDA on soft BRM · toy soft MDP | slope of stability/excess MSBE | No clear effect |
| vary network width H in {8,16,32} (ablation) | SGDA on soft BRM · toy soft MDP | rate of excess MSBE | No clear effect |
Experimental setup
Builds on: soft Bellman residual minimization
Closest papers by meaning
- Computationally Efficient RL under Linear Bellman Completeness for Deterministic Dynamics (ICLR 2025 · similarity 0.88)
- Trajectory Bellman Residual Minimization: A Simple Value-Based Method for LLM Reasoning (NeurIPS 2025 · similarity 0.86)
- Bellman Residual Minimization for Control: Geometry, Stationarity, and Convergence (NeurIPS 2026 · similarity 0.86)
- Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Gradual Transition from Bellman Optimality Operator to Bellman Operator in Online Reinforcement Learning (ICML 2025 · similarity 0.85)
- Neural Network Approximation for Pessimistic Offline Reinforcement Learning (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
