Research map / Video / Video-language understanding
Paxion: Patching Action Knowledge in Video-Language Foundation Models
Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, Heng Ji
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DVDM (VAC and ATM) to the Perceiver Knowledge Patcher (ablation) | InternVideo KP-Perceiver · ActionBench average | accuracy (%): 67.7 -> 85.9 | Improved |
| Add DVDM (VAC and ATM) to the Perceiver Knowledge Patcher (ablation) | CLIP-ViP KP-Perceiver · ActionBench average | accuracy (%): 59.2 -> 75.4 | Improved |
| Add DVDM (VAC and ATM) to the Perceiver Knowledge Patcher (ablation) | Singularity KP-Perceiver · ActionBench average | accuracy (%): 55.3 -> 73.5 | Improved |
| Replace the Transformer Knowledge Patcher with a Perceiver (ablation) | InternVideo Knowledge Patcher · ActionBench average | accuracy (%): 64.3 -> 67.7 | Improved |
| Fuse the action patcher with the frozen backbone (ablation) | Paxion · Moments-In-Time, zero-shot cross-domain | validation accuracy (%): 4.3 -> 21.6 | Improved |
| Fuse the action patcher with the frozen backbone (ablation) | Paxion · Temporal-Kinetic, zero-shot cross-domain | validation accuracy (%): 16.3 -> 49.7 | Improved |
| Ensemble Paxion predictions with backbone predictions (ablation) | Paxion · Moments-In-Time, zero-shot cross-domain | validation accuracy (%): 21.6 -> 23.9 | Improved |
| Ensemble Paxion predictions with backbone predictions (ablation) | Paxion · Temporal-Kinetic, zero-shot cross-domain | validation accuracy (%): 49.7 -> 58.1 | Improved |
| Use the Knowledge Fuser rather than a pooled-feature alpha blend | Paxion vs Side-Tuning · SSv2-template | Recall@1 (%): 22.2 -> 26.9 | Improved |
| Use DVDM-trained action knowledge rather than VTC-only patching (ablation) | Paxion · Temporal-SSv2 | Recall@1 (%): 42.5 -> 51.2 | Improved |
| Use the fused representation instead of the action patcher alone (ablation) | Paxion · SSv2-label | video-to-text Recall@1 (%) | Improved |
Closest papers by meaning
- Open-Vocabulary Video Relation Extraction (AAAI 2024 · similarity 0.88)
- Kronecker Mask and Interpretive Prompts are Language-Action Video Learners (ICLR 2025 · similarity 0.88)
- LLaVAction: evaluating and training multi-modal large language models for action understanding (ICLR 2026 · similarity 0.87)
- FAVOR-Bench: A Comprehensive Benchmark for Fine-Grained Video Motion Understanding (NeurIPS 2025 · similarity 0.87)
- VideoWorld: Exploring Knowledge Learning from Unlabeled Videos (CVPR 2025 · similarity 0.87)
- Can't Make an Omelette Without Breaking Some Eggs: Plausible Action Anticipation Using Large Video-Language Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
