AI Paper Atlas

Research map / Video / Video-language understanding

Paxion: Patching Action Knowledge in Video-Language Foundation Models

Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, Heng Ji

NeurIPS 2023 · Video › Video-language understanding · approach: Long video understanding · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add DVDM (VAC and ATM) to the Perceiver Knowledge Patcher (ablation)InternVideo KP-Perceiver · ActionBench averageaccuracy (%): 67.7 -> 85.9Improved
Add DVDM (VAC and ATM) to the Perceiver Knowledge Patcher (ablation)CLIP-ViP KP-Perceiver · ActionBench averageaccuracy (%): 59.2 -> 75.4Improved
Add DVDM (VAC and ATM) to the Perceiver Knowledge Patcher (ablation)Singularity KP-Perceiver · ActionBench averageaccuracy (%): 55.3 -> 73.5Improved
Replace the Transformer Knowledge Patcher with a Perceiver (ablation)InternVideo Knowledge Patcher · ActionBench averageaccuracy (%): 64.3 -> 67.7Improved
Fuse the action patcher with the frozen backbone (ablation)Paxion · Moments-In-Time, zero-shot cross-domainvalidation accuracy (%): 4.3 -> 21.6Improved
Fuse the action patcher with the frozen backbone (ablation)Paxion · Temporal-Kinetic, zero-shot cross-domainvalidation accuracy (%): 16.3 -> 49.7Improved
Ensemble Paxion predictions with backbone predictions (ablation)Paxion · Moments-In-Time, zero-shot cross-domainvalidation accuracy (%): 21.6 -> 23.9Improved
Ensemble Paxion predictions with backbone predictions (ablation)Paxion · Temporal-Kinetic, zero-shot cross-domainvalidation accuracy (%): 49.7 -> 58.1Improved
Use the Knowledge Fuser rather than a pooled-feature alpha blendPaxion vs Side-Tuning · SSv2-templateRecall@1 (%): 22.2 -> 26.9Improved
Use DVDM-trained action knowledge rather than VTC-only patching (ablation)Paxion · Temporal-SSv2Recall@1 (%): 42.5 -> 51.2Improved
Use the fused representation instead of the action patcher alone (ablation)Paxion · SSv2-labelvideo-to-text Recall@1 (%)Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy