Research map / Video / Video segmentation and tracking
Truncated Emphatic Temporal Difference Methods for Prediction and Control
Shangtong Zhang, Shimon Whiteson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| truncate the followon trace at n=4 rather than use the full trace (ablation) | Truncated ETD · Baird's counterexample prediction | prediction error | Improved |
| truncate the followon trace at n=2 rather than n=4 (ablation) | Truncated ETD · Baird's counterexample prediction, π(dashed|s) ≥ 0.04 | prediction error | No clear effect |
| increase truncation length from n=4 to n=8 (ablation) | Truncated ETD · Baird's counterexample prediction | variance | Hurt |
| replace hard truncation with soft truncation β=0.8 (ablation) | Truncated ETD · Baird's counterexample prediction | variance | Hurt |
| use any tested truncation length or β variant with π(dashed|s)=0 (ablation) | Truncated ETD · Baird's counterexample prediction | prediction progress | No clear effect |
| truncate the followon trace at n=4 (ablation) | Truncated Emphatic Expected SARSA · Baird's counterexample control, fixed behavior policy | control learning progress | Improved |
| replace n=4 hard truncation with full trace n=∞ (ablation) | Truncated Emphatic Expected SARSA · Baird's counterexample control, fixed behavior policy | control learning progress | Hurt |
| replace hard truncation with soft truncation β=0.8 (ablation) | Truncated Emphatic Expected SARSA · Baird's counterexample control, fixed behavior policy | variance | Hurt |
| use a changing behavior policy instead of the fixed behavior policy (ablation) | Truncated Emphatic Expected SARSA · Baird's counterexample control | variance | Improved |
| truncate the followon trace at n=4 instead of n=∞ (ablation) | Truncated Emphatic Expected SARSA · CartPole | episodic return | Improved |
| increase truncation length from n=2 to n=4 (ablation) | Truncated Emphatic Expected SARSA · CartPole | episodic return | Mixed |
| use hard truncation n=4 instead of soft truncation β=0.2 (ablation) | Truncated Emphatic Expected SARSA · CartPole | variance: $1.8 \times 10^4$ -> $7 \times 10^3$ | Improved |
Closest papers by meaning
- An Emphatic Approach to the Problem of Off-policy Temporal-Difference Learning (JMLR 2016 · similarity 0.84)
- Generalized Emphatic Temporal Difference Learning: Bias-Variance Analysis (AAAI 2016 · similarity 0.83)
- PER-ETD: A Polynomially Efficient Emphatic Temporal Difference Learning Method (ICLR 2022 · similarity 0.83)
- An Analysis of Quantile Temporal-Difference Learning (ICML 2025 · similarity 0.80)
- On-Line Learning of Linear Dynamical Systems: Exponential Forgetting in Kalman Filters (AAAI 2019 · similarity 0.80)
- Temporal Difference Learning for Model Predictive Control (ICML 2022 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
