Research map / Reinforcement learning / Policy gradient and actor-critic
HodgeFlow Policy Search by Topologically Dissecting Temporal-Difference Signals in Non-Markovian Environments
Zuyuan Zhang, Sizhe Tang, Tian Lan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| project TD errors onto their integrable component (HFPS) instead of standard TD learning | HFPS · Nonmarkov LunarLander | Final@T return: 145.6±12.9 vs DQN 10.1±34.3; DRQN 169.1±3.1 | Mixed |
| use HFPS instead of recurrent baselines | HFPS · Nonmarkov Pendulum | Final@T return: -106.2±121.9 vs DRQN -235.8±47.8 | Improved |
| use HFPS instead of recurrent baselines | HFPS · Nonmarkov Acrobot | Final@T return: -82.3±9.7 vs DRQN -500.0±0.0 | Improved |
| use HFPS instead of DQN | HFPS · Nonmarkov PointMass | Final@T return: -207.1±49.8 vs DQN -673.6±366.5 | Improved |
| use linear HFPS instead of linear TD in random-feature MDP | HFPS · Random-feature MDP (50 states) | MSVE | Improved |
| make the ring MDP non-integrable by perturbing one edge reward | HFPS · Tabular ring MDP | residual norm | Hurt |
Closest papers by meaning
- Composite Flow Matching for Reinforcement Learning with Shifted-Dynamics Data (NeurIPS 2025 · similarity 0.86)
- Diffusion Spectral Representation for Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Mitigating Partial Observability in Sequential Decision Processes via the Lambda Discrepancy (NeurIPS 2024 · similarity 0.86)
- Tackling Non-Stationarity in Reinforcement Learning via Causal-Origin Representation (ICML 2024 · similarity 0.86)
- A Temporal Difference Method for Stochastic Continuous Dynamics (NeurIPS 2025 · similarity 0.86)
- Geometric Active Exploration in Markov Decision Processes: the Benefit of Abstraction (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
