Research map / Video / Video-language understanding
Free-VSC: Free Semantics from Visual Foundation Models for Unsupervised Video Semantic Compression
Yuan Tian, Guo Lu, Guangtao Zhai
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace VFM semantic objective with single MAE SSL objective (60K videos) (ablation) | Free-VSC · UCF101 @0.02bpp (TSM) | Top1 accuracy: over 5% accuracy drop | Hurt |
| Add a second SSL objective and use all 230K videos instead of 1 SSL on 60K (ablation) | Free-VSC with SSL objective · UCF101 @0.02bpp (TSM) | Top1 accuracy: about +2% (still about 2% below VFM model) | Improved |
| Add a second VFM (DINOv2 with VideoMAEv2) (ablation) | Free-VSC · UCF101 @0.02bpp (TSM) | Top1 accuracy: 83.11 -> 84.11 | Improved |
| Add third VFM SSL-Swin (ablation) | Free-VSC · UCF101 @0.02bpp (TSM) | Top1 accuracy: 84.11 -> 85.07 | Improved |
| Add VFM-specific prompts to the shared alignment transformer (ablation) | Prom-SAL · UCF101 @0.02bpp (TSM) | Top1 accuracy: 80.92 -> 85.07 | Improved |
| Share Prom-SAL transformer parameters across VFMs (ablation) | Prom-SAL · UCF101 @0.02bpp (TSM) | Top1 accuracy: 83.01 -> 85.07 | Improved |
| Use vanilla Transformer rather than 3D CNN for the alignment layer (ablation) | Prom-SAL · UCF101 @0.02bpp (TSM) | Top1 accuracy: 79.56 -> 85.07 | Improved |
| Use both coarse and fine VFM features instead of one level (ablation) | Free-VSC · UCF101 @0.02bpp (TSM) | Top1 accuracy: 84.28 -> 85.07 (vs fine only) | Improved |
| Use video-adaptive dynamic and region-wise trajectory over static trajectory (ablation) | entropy model · UCF101 @0.02bpp (TSM) | BD-Rate: bitcost +21.7% without both designs | Improved |
| Use trajectory-based entropy model instead of motion compensation or ConvLSTM (ablation) | inter-frame entropy model · UCF101 @0.02bpp (TSM) | Top1 accuracy: about 1.5% and 2.2% accuracy drops for motion and ConvLSTM | Improved |
| Increase number of trajectories S from 5 to 10 (ablation) | entropy model · UCF101 @0.02bpp (TSM) | BD-Rate: 0% -> -1.8% | No clear effect |
| Apply VFM semantic learning to DCVC-DC and SMC (ablation) | DCVC-DC and SMC · UCF101 @0.02bpp (TSM) | Top1 accuracy | Improved |
Closest papers by meaning
- One Token per Highly Selective Frame: Towards Extreme Compression for Long Video Understanding (NeurIPS 2025 · similarity 0.87)
- Towards Unified Human Perception and Machine Understanding: Token Flow Guided Compression Framework (CVPR 2026 · similarity 0.87)
- Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity (ICLR 2026 · similarity 0.86)
- PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models (CVPR 2025 · similarity 0.85)
- Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding (CVPR 2025 · similarity 0.85)
- Less Is More, but Where? Dynamic Token Compression via LLM-Guided Keyframe Prior (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
