Research map / Label-efficient and robust learning / Domain adaptation and generalization
Image-to-video Adaptation with Outlier Modeling and Robust Self-learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage image-to-video adaptation method that adds class-specific outlier neurons (with batch nuclear norm maximization and a pseudo outlier-class loss) and then selects reliable pseudo-labels via a label-propagation-consistency metric before FixMatch-based video self-learning.
Problem
Image-to-video adaptation must bridge the domain gap between labeled source images and unlabeled target videos, including the modality gap (no temporal info in images) and outlier target frames lacking semantic content, while pseudo labels from the image model are noisy.
Approach
Extend the classifier with class-specific outlier neurons trained by BNM and a pseudo outlier-class loss that treats source images of other classes as outliers, and filter pseudo labels with a label-propagation consistency metric (memory-based forward-backward nearest-neighbour cycle) before semi-supervised video training.
Key results
- Achieves 78.1% on B->U, a 5.5-point gain over CycDA (72.6%).
- Achieves 66.3% on E->H, a 4.3-point gain over CycDA (62.0%), and 99.0% on S->U, comparable to CycDA's 99.1%.
- Stage-1 BNM raises E->H accuracy from 41.7 (source only) to 54.6; adding the pseudo outlier-class loss gives 55.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add outlier classes and BNM loss (ablation) | Image-level adaptation model · S→U | accuracy: 89.1 -> 96.5 | Helped |
| add outlier classes and BNM loss (ablation) | Image-level adaptation model · B→U | accuracy: 68.2 -> 77.0 | Helped |
| add outlier classes and BNM loss (ablation) | Image-level adaptation model · E→H | accuracy: 41.7 -> 54.6 | Helped |
| add pseudo outlier-class loss to SO+BNM (ablation) | Image-level adaptation model · S→U | accuracy: 96.5 -> 96.9 | Helped |
| add pseudo outlier-class loss to SO+BNM (ablation) | Image-level adaptation model · B→U | accuracy: 77.0 -> 77.1 | Helped |
| add pseudo outlier-class loss to SO+BNM (ablation) | Image-level adaptation model · E→H | accuracy: 54.6 -> 55.3 | Helped |
| select pseudo-labeled videos with label propagation consistency and FixMatch instead of using all stage-1 pseudo labels (ablation) | Stage-2 I3D video model · S→U | accuracy: 98.5 -> 99.0 | Helped |
| select pseudo-labeled videos with label propagation consistency and FixMatch instead of using all stage-1 pseudo labels (ablation) | Stage-2 I3D video model · B→U | accuracy: 77.5 -> 78.1 | Helped |
| select pseudo-labeled videos with label propagation consistency and FixMatch instead of using all stage-1 pseudo labels (ablation) | Stage-2 I3D video model · E→H | accuracy: 64.0 -> 66.3 | Helped |
Surprising findings
- Stage-2 gains from the label-propagation selection are largest on E->H (64.0 to 66.3), and the method is essentially on par with CycDA on S->U (99.0 vs 99.1).
Limitations
- Reports results on only three image-to-video benchmarks.
- Selection-ratio and threshold hyperparameters are not analyzed beyond lambda_poc sensitivity on E->H.
- Comparable rather than superior performance on S->U.
Experimental setup
Datasets: Stanford40 BU101 EADs UCF101 HMDB51 Kinetics
Benchmarks: Stanford40->UCF101 BU101->UCF101 EADs->HMDB51
Metrics: classification accuracy
Baselines: UnAtt SymGAN DANN+I3D HPDA CycDA source only
Builds on: FixMatch batch nuclear norm maximization CycDA I3D K-reciprocal nearest neighbours
Closest papers by meaning
- Discovering Informative and Robust Positives for Video Domain Adaptation (ICLR 2023 · similarity 0.87)
- Toward Improving Robustness and Accuracy in Unsupervised Domain Adaptation (AAAI 2025 · similarity 0.85)
- Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training (CVPR 2024 · similarity 0.85)
- Mean Teacher DETR with Masked Feature Alignment: A Robust Domain Adaptive Detection Transformer Framework (AAAI 2024 · similarity 0.84)
- Confusing Pair Correction Based on Category Prototype for Domain Adaptation under Noisy Environments (AAAI 2024 · similarity 0.84)
- Test-Time Domain Adaptation by Learning Domain-Aware Batch Normalization (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.