Research map / Reinforcement learning / Policy gradient and actor-critic
Expected Eligibility Traces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use expected eligibility traces instead of instantaneous traces | TD(λ) value learning · Multi-chain | RMSE | Helped |
| Use expected eligibility traces instead of instantaneous traces | TD(λ) value learning · Open world grid | learning speed and stability | Helped |
| Use expected traces rather than accumulating traces | Deep Q(λ) · Pong and Ms. Pac-Man | learning speed | Helped |
| Replace instantaneous traces with exact expected traces under Markov states | TD(λ) · Markov-state temporal-difference learning | update variance | Helped |
| Estimate expected traces by a running empirical mean | TD(λ) · Markov-state temporal-difference learning | update variance | Helped |
| Set η=1, yielding instantaneous traces (ablation) | ET(λ, η) · Temporal-difference learning | algorithm equivalence | No clear effect |
| Set η=0, yielding expected traces (ablation) | ET(λ, η) · Temporal-difference learning | algorithm equivalence | No clear effect |
| Vary η to interpolate between expected and instantaneous traces (ablation) | ET(λ, η) · Multi-chain, linear function approximation | RMSE | Mixed |
| Vary λ and step size (ablation) | ET(λ) · Multi-chain with 32 parallel chains | prediction error | No clear effect |
Closest papers by meaning
- Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysis (NeurIPS 2023 · similarity 0.90)
- From Past to Future: Rethinking Eligibility Traces (AAAI 2024 · similarity 0.90)
- Sequence Compression Speeds Up Credit Assignment in Reinforcement Learning (ICML 2024 · similarity 0.87)
- Do We Need to Verify Step by Step? Rethinking Process Supervision from a Theoretical Perspective (ICML 2025 · similarity 0.86)
- Reward-Aware Proto-Representations in Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.