Research map / Label-efficient and robust learning / Domain adaptation and generalization
TAMT: Temporal-Aware Model Tuning for Cross-Domain Few-Shot Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TAMT, a decoupled pre-train-on-source then fine-tune-on-target paradigm for cross-domain few-shot action recognition, using parameter-efficient temporal-aware adapters and a long-short temporal covariance moment head on a VideoMAE backbone.
Problem
Cross-domain few-shot action recognition (CDFSAR) requires transferring from a labeled source video dataset to target domains with few labels; existing joint source-target training methods require costly retraining for each target and use pre-trained models inefficiently.
Approach
Decouple training into source pre-training (SSL then supervised) followed by target fine-tuning with a hierarchical temporal tuning network: local temporal-aware adapters on frozen intermediate features plus a global moment-based head (first-order GAP and efficient long-short temporal covariance) in a metric-based few-shot pipeline.
Key results
- Averaged over five target datasets with K-100 source, TAMT reaches 57.91% vs. 26.76% for CDFSL-V (+31.15%) at 112x112 with ViT-S; TAMT* at 224x224 reaches 65.75%.
- With K-400 source, TAMT averages 68.37% vs. 47.24% for CDFSL-V (+21.13%).
- Compared with full fine-tuning, TAMT uses about 0.1x GPU memory (1.9G vs 17.5G) and 0.1x trainable parameters (2.8M vs 29.9M), with 0.7x training time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TAMT instead of CDFSL-V with K-400 source pretraining | CDFSAR · Average across HMDB, SSV2, Diving, UCF, and RareAct | 5-way 5-shot accuracy: 47.24 -> 68.37(+21.13) | Helped |
| Use TAMT instead of CDFSL-V with K-100 source pretraining | CDFSAR · Average across HMDB, SSV2, Diving, UCF, and RareAct | 5-way 5-shot accuracy: 26.76 -> 57.91(+31.15) | Helped |
| Use TAMT instead of CDFSL-V | CDFSAR with SSV2 source · Average over available target datasets | 5-way 5-shot accuracy: 29.00 -> 57.02(+28.02) | Helped |
| Use TAMT instead of CDFSL-V | CDFSAR with Diving source · Average over available target datasets | 5-way 5-shot accuracy: 28.00 -> 45.92(+17.92) | Helped |
| Use TAMT instead of CDFSL-V | CDFSAR with UCF source · Average over available target datasets | 5-way 5-shot accuracy: 26.82 -> 39.93(+13.11) | Helped |
| Add supervised learning after SSL during pretraining (ablation) | TAMT · SSV2 | 5-way 5-shot accuracy: 34.91 -> 40.45 | Helped |
| Add supervised learning after SSL during pretraining (ablation) | TAMT · Diving | 5-way 5-shot accuracy: 27.15 -> 28.09 | Helped |
| Add supervised learning after SSL during pretraining (ablation) | TAMT · UCF | 5-way 5-shot accuracy: 88.41 -> 94.69 | Helped |
| Use TAMT tuning instead of full fine-tuning with SSL+SL pretraining (ablation) | TAMT · SSV2, Diving, and UCF | 5-way 5-shot accuracy: 55.99 -> 59.18; 42.85 -> 45.18; 94.95 -> 95.92 | Mixed |
| Add GTMT instead of using GAP without an adapter (ablation) | HTTN · SSV2, Diving, and UCF | 5-way 5-shot accuracy: 40.45 -> 53.41; 28.09 -> 42.87; 94.69 -> 94.97 | Helped |
| Add TAA to GTMT (ablation) | HTTN · SSV2, Diving, and UCF | 5-way 5-shot accuracy: 53.41 -> 59.18; 42.87 -> 45.18; 94.97 -> 95.92 | Helped |
| Use ELSTC instead of TCov with TAA (ablation) | HTTN · SSV2, Diving, and UCF | 5-way 5-shot accuracy: 56.72 -> 58.56; 43.68 -> 43.90; 95.16 -> 95.37 | Helped |
Surprising findings
- Full fine-tuning overfits on SSV2 after about 30 epochs while TAMT keeps validation performance stable.
Limitations
- Pre-training on source still requires a separate SSL+SL stage; SSL alone with single-stage pretraining made FFT slightly better than TAMT.
- Results on some targets (e.g., SSV2 with K-100 source) lag behind the 224-resolution variants, and performance depends on resolution.
Experimental setup
Datasets: Kinetics-400 Kinetics-100 Something-Something V2 Diving48 UCF-101 HMDB51 RareAct
Benchmarks: HMDB51 SSV2 Diving48 UCF-101 RareAct
Metrics: 5-way 5-shot accuracy 5-way 1-shot accuracy GPU memory trainable parameters training time
Baselines: SEEN CDFSL-V DMSD STARTUP++ DD++ STRM HyRSM Adapter SSF TAM ST-Adapter FFT
Builds on: VideoMAE ST-Adapter CDFSL-V SEEN
Closest papers by meaning
- Learning Causal Domain-Invariant Temporal Dynamics for Few-Shot Action Recognition (ICML 2024 · similarity 0.87)
- Manta: Enhancing Mamba for Few-Shot Action Recognition of Long Sub-Sequence (AAAI 2025 · similarity 0.86)
- Frame Order Matters: A Temporal Sequence-Aware Model for Few-Shot Action Recognition (AAAI 2025 · similarity 0.86)
- Temporal Alignment-Free Video Matching for Few-shot Action Recognition (CVPR 2025 · similarity 0.85)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.85)
- Random Registers for Cross-Domain Few-Shot Learning (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.