Research map / Label-efficient and robust learning / Domain adaptation and generalization
Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UNITE, a three-stage unsupervised video domain adaptation pipeline that uses CLIP as a teacher for masked distillation pre-training on target videos and collaborative pseudolabeling during masked self-training.
Problem
Unsupervised domain adaptation for video action recognition, where labeled source videos and unlabeled target videos differ in distribution and the goal is to improve target-domain accuracy.
Approach
Combines UMT-style masked feature distillation from a frozen CLIP image teacher on target videos (stage 1), source supervised fine-tuning (stage 2), and collaborative self-training where CLIP zero-shot predictions and the video student jointly produce MatchOrConf pseudolabels applied to masked target videos (stage 3).
Key results
- On Daily-DA, UNITE averages 59.2% top-1 vs 53.3% for zero-shot CLIP and 51.4% for DALL-V (LB/UB rows aside).
- On Sports-DA, UNITE reaches 94.0% average accuracy, versus 89.9% for zero-shot CLIP and 82.3% for DALL-V.
- On UCF->HMDB, UNITE achieves 95.0% on U->H (state of the art), while UNITE averages 93.8% on UCF<->HMDB versus 94.6% for UDAVT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add collaborative self-training to source-only training (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 49.6 -> 60.8 | Helped |
| add target UMT pre-training to source-only training (ablation) | UNITE · Daily-DA, H→A | accuracy (%): 40.4 -> 43.8 | Helped |
| combine target UMT pre-training and collaborative self-training (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 49.6 -> 67.9 | Helped |
| use target videos only rather than source videos only for UMT pre-training (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 60.0 -> 67.9 | Helped |
| include both source and target videos rather than target videos only in UMT pre-training (ablation) | UNITE · Daily-DA, H→A | accuracy (%): 48.0 -> 45.2 | Hurt |
| use masked rather than unmasked target videos for target CE loss (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 54.6 -> 67.9 | Helped |
| use MatchOrConf rather than Cons pseudolabeling (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 57.5 -> 67.9 | Helped |
| use MatchOrConf rather than ConsOrConf pseudolabeling (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 56.3 -> 67.9 | Helped |
| use student and teacher predictions together rather than zero-shot CLIP only (ablation) | UNITE · Daily-DA, H→A | accuracy (%): 34.7 -> 48.0 | Helped |
| add a masked-consistency constraint to MatchOrConf (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 67.9 -> 60.8 | Hurt |
| retain source-domain classification loss during self-training (ablation) | UNITE · Daily-DA, A→H | accuracy (%): 42.1 -> 67.9 | Helped |
| set MatchOrConf confidence threshold to γ = 0.5 rather than γ = 0.1 (ablation) | UNITE · Daily-DA, H→A | accuracy (%): 48.0 -> 51.9 | Helped |
Surprising findings
- Source classification loss is essential during masked self-training; removing it drops ARID->HMDB accuracy from 48.0% to 33.7%.
- Pre-training on source plus target videos during UMT was suboptimal compared with target-only pre-training (45.2% vs 48.0% on H->A).
Limitations
- Pseudolabel confidence threshold must be tuned per domain shift; results vary substantially across shifts (Table 9).
- Performance can degrade on some shifts (e.g., M->A below source-only), and some classes decline after self-training.
- Uses ImageNet-free self-supervised init from Kinetics-710 rather than supervised pre-training, so comparisons with prior work using supervised init are not direct.
Experimental setup
Datasets: Daily-DA Sports-DA UCF-HMDB_full Kinetics-710
Benchmarks: Daily-DA Sports-DA UCF-HMDB_full
Metrics: top-1 accuracy
Baselines: DANN MK-MMD TA3N CO2A UDAVT ATCoN EXTERN DALL-V CLIP zero-shot
Builds on: Unmasked Teacher (UMT) PACMAC MatchOrConf UMT masked distillation MIC
Closest papers by meaning
- Unsupervised Video Domain Adaptation for Action Recognition: A Disentanglement Perspective (NeurIPS 2023 · similarity 0.91)
- Multi-Modal Domain Adaptation for Fine-Grained Action Recognition (CVPR 2020 · similarity 0.89)
- AutoLabel: CLIP-Based Framework for Open-Set Video Domain Adaptation (CVPR 2023 · similarity 0.87)
- Modality-Collaborative Test-Time Adaptation for Action Recognition (CVPR 2024 · similarity 0.87)
- Image-to-video Adaptation with Outlier Modeling and Robust Self-learning (AAAI 2025 · similarity 0.85)
- Can We Evaluate Domain Adaptation Models Without Target-Domain Labels? (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.