Research map / Reinforcement learning / Policy gradient and actor-critic
Policy Evaluation and Temporal-Difference Learning in Continuous Time and Space: A Martingale Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace mean-square TD error (MSTDE) objective with martingale loss function for policy evaluation | policy evaluation with diffusion dynamics · Brownian motion toy example (J=E[X_1|X_t=x]) | convergence to true value function: MSTDE converges to theta=-3/2 vs ML converges to true theta=0 | Helped |
| Use TD(0)/TD(lambda)-style martingale orthogonality conditions instead of MSTDE | online policy evaluation · Brownian motion examples | convergence to correct value: CTD(0) and CTD(1) converge to correct values | Helped |
| Online MSTDE one-step update | MSTDE algorithms · Brownian motion examples | convergence to correct value | Hurt |
Closest papers by meaning
- A Non-asymptotic Analysis of Non-parametric Temporal-Difference Learning (NeurIPS 2022 · similarity 0.87)
- An Analysis of Quantile Temporal-Difference Learning (ICML 2025 · similarity 0.84)
- The surprising efficiency of temporal difference learning for rare event prediction (NeurIPS 2024 · similarity 0.84)
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.82)
- Finite Sample Analysis of Linear Temporal Difference Learning with Arbitrary Features (NeurIPS 2025 · similarity 0.82)
- When to Sense and Control? A Time-adaptive Approach for Continuous-Time RL (NeurIPS 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.