Research map / Video / Action recognition
Boosting Video Representation Learning With Multi-Faceted Integration
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace intra-facet L2 loss with contrastive intra-facet NCE loss (ablation) | Visual-semantic embedding for video representation learning · Average of six facets | accuracy: 56.6 -> 59.9 | Helped |
| Add inter-facet supervision from the four video datasets (ablation) | intra_nce video representation · Average of six facets | accuracy: 59.9 -> 60.2 | Helped |
| Add inter-facet supervision from video and image datasets (ablation) | intra_nce video representation · Average of six facets | accuracy: 59.9 -> 60.9 | Helped |
| Use video-only inter-facet supervision instead of intra-facet supervision alone (ablation) | intra_nce video representation · Action, event, interaction, and sport facets | accuracy: 55.6 -> 56.2; 25.9 -> 26.2; 42.4 -> 44.1; 65.3 -> 65.7 | Helped |
| Use video-only inter-facet supervision (ablation) | intra_nce video representation · Object and scene facets | accuracy: 74.7 -> 74.6; 95.6 -> 94.8 | Hurt |
| Add inter-facet supervision from image datasets as well as video datasets (ablation) | intra_nce video representation · Object and scene facets | accuracy: 74.7 -> 76.7; 95.6 -> 96.9 | Helped |
| Add multi-attention projection (ablation) | intra_nce+inter_all · Average of six facets | accuracy: 60.9 -> 62.6 | Helped |
| Train on four facets jointly with classification instead of one facet (ablation) | Video representation learning by classification · Average of six facets | accuracy: 52.2 -> 57.2 | Helped |
| Use four separate classifier heads rather than one shared classifier head (ablation) | 4facets classification model · Average of six facets | accuracy: 57.2 -> 57.1 | No clear effect |
| Fine-tune successively on MIT, SSV2, then S1M rather than train across facets holistically (ablation) | K700-pretrained network · Action and sport facet validation | accuracy: 28% -> 68.8%; 57.6% -> 35.7% | Mixed |
| Fine-tune MUFI representation for downstream action recognition | MUFI pretrained ResNet-50 · UCF101 and HMDB51 | accuracy: 97.0 -> 98.1; 75.7 -> 80.9 | Helped |
| Pretrain with MUFI rather than the compared state-of-the-art initialization | Downstream video captioning model · MSVD | CIDEr-D: 95.2 -> 101.5 | Helped |
Closest papers by meaning
- Unsupervised Learning From Video With Deep Neural Embeddings (CVPR 2020 · similarity 0.85)
- MoTE: Reconciling Generalization with Specialization for Visual-Language to Video Knowledge Transfer (NeurIPS 2024 · similarity 0.85)
- ClusterFit: Improving Generalization of Visual Representations (CVPR 2020 · similarity 0.85)
- MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence (NeurIPS 2025 · similarity 0.84)
- Efficient Transfer Learning for Video-language Foundation Models (CVPR 2025 · similarity 0.84)
- Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.