AI Paper Atlas

Research map / Reinforcement learning / Visual and representation RL

Semantic Visual Navigation by Watching YouTube Videos

Matthew Chang, Arjun Gupta, Saurabh Gupta

NeurIPS 2020 · Reinforcement learning › Visual and representation RL · approach: Vision-based robot RL · 38 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
learn value function from YouTube videos with Q-learning on pseudo-labeled actions instead of end-to-end RL with dense reward and far more interactionhierarchical ObjectGoal policy · Gibson tiny val Oracle StopSPL: 0.29 -> 0.53Improved
learn value via Q-learning from videos instead of behavior cloning on same pseudo-labeled videosnavigation policy · Gibson tiny val Oracle StopSPL: 0.25 -> 0.53Improved
use real YouTube videos instead of synthetic videos for VLVVLV · Gibson tiny val Oracle StopSPL: 0.48 -> 0.53Improved
add VLV Q-learning objective jointly to strong map-supervised value trainingStrong Supervision Values · Gibson tiny val Oracle StopSPL: 0.55 -> 0.57Improved
use true actions rather than inverse-model pseudo-labels (ablation)VLV base · Gibson train envs Oracle StopSPL: 0.40 -> 0.41No clear effect
use ground-truth detections rather than Mask RCNN (ablation)VLV base · Gibson train envs Oracle StopSPL: 0.40 -> 0.40No clear effect
use optimal trajectories rather than noisy ones (ablation)VLV base · Gibson train envs Oracle StopSPL: 0.40 -> 0.43No clear effect
add Mask RCNN score when sampling short-term goals (ablation)VLV base · Gibson train envs Oracle StopSPL: 0.40 -> 0.46Improved
use 360 degree images at training time (ablation)VLV base · Gibson train envs Oracle StopSPL: 0.40 -> 0.47Improved
Q-learning with pseudo-labeled actions instead of policy evaluation without actions (ablation)value learning · branching environmentreaches near goalImproved
fine-tune behavior cloning policy with dense-reward RLBehavior Cloning · Gibson tiny val Oracle StopSPL: 0.36 -> 0.24Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy