AI Paper Atlas

Research map / Video / Video segmentation and tracking

Truncated Emphatic Temporal Difference Methods for Prediction and Control

Shangtong Zhang, Shimon Whiteson

NeurIPS 2022 · Video › Video segmentation and tracking · approach: Stochastic video prediction

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
truncate the followon trace at n=4 rather than use the full trace (ablation)Truncated ETD · Baird's counterexample predictionprediction errorImproved
truncate the followon trace at n=2 rather than n=4 (ablation)Truncated ETD · Baird's counterexample prediction, π(dashed|s) ≥ 0.04prediction errorNo clear effect
increase truncation length from n=4 to n=8 (ablation)Truncated ETD · Baird's counterexample predictionvarianceHurt
replace hard truncation with soft truncation β=0.8 (ablation)Truncated ETD · Baird's counterexample predictionvarianceHurt
use any tested truncation length or β variant with π(dashed|s)=0 (ablation)Truncated ETD · Baird's counterexample predictionprediction progressNo clear effect
truncate the followon trace at n=4 (ablation)Truncated Emphatic Expected SARSA · Baird's counterexample control, fixed behavior policycontrol learning progressImproved
replace n=4 hard truncation with full trace n=∞ (ablation)Truncated Emphatic Expected SARSA · Baird's counterexample control, fixed behavior policycontrol learning progressHurt
replace hard truncation with soft truncation β=0.8 (ablation)Truncated Emphatic Expected SARSA · Baird's counterexample control, fixed behavior policyvarianceHurt
use a changing behavior policy instead of the fixed behavior policy (ablation)Truncated Emphatic Expected SARSA · Baird's counterexample controlvarianceImproved
truncate the followon trace at n=4 instead of n=∞ (ablation)Truncated Emphatic Expected SARSA · CartPoleepisodic returnImproved
increase truncation length from n=2 to n=4 (ablation)Truncated Emphatic Expected SARSA · CartPoleepisodic returnMixed
use hard truncation n=4 instead of soft truncation β=0.2 (ablation)Truncated Emphatic Expected SARSA · CartPolevariance: $1.8 \times 10^4$ -> $7 \times 10^3$Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy