AI Paper Atlas

Research map / Video / Video segmentation and tracking

Look Ma, No Hands! Agent-Environment Factorization of Egocentric Videos

Matthew Chang, Aditya Prakash, Saurabh Gupta

NeurIPS 2023 · Video › Video segmentation and tracking · approach: Layered video decomposition and dynamics · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use past-frame attention (VIDM) instead of single-frame LDM inpaintinginpainting model · EPIC held-out inpaintingPSNR: 28.29 -> 32.26Improved
Use past-frame attention instead of single-frame LDMinpainting model · EPIC held-out inpaintingFID: 27.29 -> 10.37Improved
Fine-tune single-frame LDM on EPIC imagesLatent Diffusion · EPIC held-out inpaintingPSNR: 28.29 -> 28.27No clear effect
Use VIDM with 3 frames history instead of DLFormerinpainting model · EPIC held-out inpaintingruntime: 106.4s / image -> ~13.6s / imageImproved
Black out the agent mask instead of raw imageMask R-CNN · VISOR active objects, Mask R-CNNAR_all@5: 0.263 -> 0.236Hurt
Inpaint agent with single-frame LDMMask R-CNN · VISOR active objects, Mask R-CNNAR_all@5: 0.263 -> 0.259Mixed
Inpaint agent with VIDM (no factorization)Mask R-CNN · VISOR active objects, Mask R-CNNAR_all@1: 0.137 -> 0.163Improved
Use both raw and VIDM-inpainted images (factorization) (ablation)Mask R-CNN · VISOR active objects, Mask R-CNNAR_all@5: 0.268 -> 0.379Improved
Add ground-truth environment image as input3D hand-held object reconstruction · ObManF1@5mm: 0.41 -> 0.48Improved
Add ground-truth environment image as input3D hand-held object reconstruction · ObManF1@10mm: 0.62 -> 0.68Improved
Add ground-truth environment image as input3D hand-held object reconstruction · ObManchamfer distance: 1.23 -> 1.06Improved
Inpaint hands with VIDM instead of maskingaffordance model · Region of Interaction 0% slackAverage Precision: 47.50 -> 50.77Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy