AI Paper Atlas

Research map / 3D vision / Human pose estimation

Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens

Elad Ben Avraham, Roei Herzig, Karttikeya Mangalam, Amir Bar, Anna Rohrbach, Leonid Karlinsky, Trevor Darrell, Amir Globerson

NeurIPS 2022 · 3D vision › Human pose estimation · approach: Temporal video pose estimation · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use SViT with annotated images from a different domainMViTv2 on SomethingElse · SomethingElseTop-1 accuracy: 63.3 -> 65.1Improved
Use only 2% of the in-domain image annotations (ablation)SViT on SomethingElse · SomethingElseTop-1 accuracy: 63.3 -> 65.6Improved
Finetune jointly on video and auxiliary images rather than finetuning only on videoSViT on SomethingElse · SomethingElseTop-1 accuracy: 64.2 -> 65.1Improved
Add object tokens and the SViT loss to the multi-task baseline (ablation)MViTv2 on SomethingElse · SomethingElseTop-1 accuracy: 63.8 -> 65.0Improved
Add frame-clip consistency loss alongside object tokens (ablation)MViTv2 on SomethingElse · SomethingElseTop-1 accuracy: 65.0 -> 65.8Improved
Increase annotated image use from 2% to 25% (ablation)SViT-ID on SomethingElse · SomethingElseTop-1 accuracy: 65.6 -> 65.6No clear effect
Increase annotated image use from 2% to 50% (ablation)SViT-ID on SomethingElse · SomethingElseTop-1 accuracy: 65.6 -> 65.7No clear effect
Use real HAOG annotations instead of random HAOGs (ablation)SViT-ID on SomethingElse · SomethingElseTop-1 accuracy: 50.6 -> 65.8Improved
Use SViT rather than MViTv2 on SSv2MViTv2 · SSv2Top-1 accuracy: 68.2 -> 69.7Improved
Use SViT rather than MViTv2 on Diving48MViTv2 · Diving48Top-1 accuracy: 73.1 -> 79.8Improved
Use SViT rather than MViTv2 on AVA-V2.2MViTv2 · AVA-V2.2mAP: 26.8 -> 28.5Improved
Apply SViT on clips with no annotated hands (ablation)MViTv2 on SomethingElse · SomethingElse, no annotated handsTop-1 accuracy: 64.6 -> 66.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy