Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Learning Causal Domain-Invariant Temporal Dynamics for Few-Shot Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DITeD, a causal generative temporal-dynamics model for few-shot action recognition that keeps domain-invariant transition/generation modules fixed during adaptation and fine-tunes only the image and domain encoders plus classifier.
Problem
Few-shot action recognition requires adapting a pretrained video model to novel classes and distribution shifts using very few labeled samples, while identifying which temporal knowledge transfers across domains.
Approach
Model action features as latent variables with a causal temporal graph whose transition and generation modules are domain-invariant and a domain index captures domain-specific variation; during adaptation only the image encoder, domain encoder and classifier are updated while the temporal dynamic modules stay fixed.
Key results
- DITeD outperforms VideoMAE in most few-shot settings on SSv2, HMDB and UCF (e.g., UCF 97.7 vs 91.0 at the lowest-shot setting per the rebuttal numbers provided).
- Fixing the temporal dynamic generation or transition modules during adaptation outperforms updating them on Sth-Else (DITeD-DG 46.0/61.3 and DITeD-DT 46.2/60.8 versus the full method).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| update the temporal dynamic generation module during adaptation (ablation) | DITeD · Sth-Else | accuracy | Hurt |
| update the temporal dynamic transition module during adaptation (ablation) | DITeD · Sth-Else | accuracy | Hurt |
| update all modules during adaptation (ablation) | DITeD · Sth-Else | accuracy | Hurt |
| fine-tune all parameters | VideoMAE · few-shot action recognition (SSv2, HMDB, UCF) | accuracy | Mixed |
Limitations
- Writing and method description judged unclear by reviewers
- Comparison backbones limited; few transformer or masked-autoencoder baselines in the main tables
Experimental setup
Datasets: Kinetics-400 Kinetics-small Sth-Else SSv2 HMDB51 UCF101
Benchmarks: Sth-Else SSv2 HMDB51 UCF101
Metrics: accuracy
Baselines: ActionCLIP XCLIP ViFi-CLIP VL Prompting VideoMAE
Builds on: CLIP ActionCLIP VideoMAE variational autoencoder
Closest papers by meaning
- Frame Order Matters: A Temporal Sequence-Aware Model for Few-Shot Action Recognition (AAAI 2025 · similarity 0.88)
- TAMT: Temporal-Aware Model Tuning for Cross-Domain Few-Shot Action Recognition (CVPR 2025 · similarity 0.87)
- Action Segmentation With Joint Self-Supervised Temporal Domain Adaptation (CVPR 2020 · similarity 0.87)
- SeFAR: Semi-supervised Fine-grained Action Recognition with Temporal Perturbation and Learning Stabilization (AAAI 2025 · similarity 0.86)
- Modality-Collaborative Test-Time Adaptation for Action Recognition (CVPR 2024 · similarity 0.86)
- Temporal Alignment-Free Video Matching for Few-shot Action Recognition (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.