Research map / Video / Video segmentation and tracking
Efficient Self-Supervised Video Hashing with Selective State Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S5VH, a Mamba-based self-supervised video hashing model with bidirectional Mamba encoder/decoder and a hash-center alignment loss as a global signal, improving accuracy, transferability, and inference scaling.
Problem
Self-supervised video hashing needs temporal modeling, but Transformer-based methods have quadratic compute and memory cost in frame number, and inter-sample contrastive signals are slow to converge.
Approach
Replaces Transformer temporal modeling with bidirectional Mamba layers (linear complexity) and adds a global hash-space signal: k-means feature clusters are mapped to well-separated, semantically consistent hash centers via an l_p-box ADMM optimization, and video hash codes are aligned to their cluster's center.
Key results
- Cross-dataset (UCF101 to HMDB51, 16-bit) GmAP of 0.0378 for S5VH versus 0.0335 for the best baseline BerVAE.
- Adding the hash center alignment loss raises GmAP by about 6% and accelerates convergence by roughly 65%, per the paper's ablation.
- Inference time fitted as linear in input length for S5VH (1.5e-3 L + 3.4e-2 ms) versus quadratic for ConMH and MCMSH.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use S5VH instead of existing hashing methods | Video hashing · UCF101 → HMDB51, 16 bits | GmAP: 0.0335 -> 0.0378 | Helped |
| use bidirectional Mamba blocks instead of forward-only or backward-only blocks (ablation) | S5VH · UCF101 and HMDB51, 16/32/64 bits | GmAP: 0.351, 0.375, 0.424; 0.079, 0.123, 0.136 (Forward Only); 0.324, 0.337, 0.390; 0.070, 0.120, 0.133 (Backward Only) | Helped |
| add contrastive learning to temporal reconstruction (ablation) | S5VH · UCF101 and HMDB51, 16/32/64 bits | GmAP: 0.138, 0.210, 0.280; 0.045, 0.075, 0.081 (TR Only) vs 0.204, 0.217, 0.290; 0.070, 0.095, 0.102 (without CL) | Helped |
| add hash center alignment loss (ablation) | S5VH · UCF101 and HMDB51 | GmAP: about 6% | Helped |
| add hash center alignment loss (ablation) | S5VH · Training convergence | convergence speed: approximately 65% | Helped |
| replace bidirectional Mamba layers with bidirectional LSTM layers (ablation) | S5VH · UCF101 and HMDB51, 16/32/64 bits | GmAP | Hurt |
| replace bidirectional Mamba layers with bidirectional RetNet layers (ablation) | S5VH · UCF101 and HMDB51, 16/32/64 bits | GmAP | Hurt |
| replace bidirectional Mamba layers with bidirectional RWKV layers (ablation) | S5VH · UCF101 and HMDB51, 16/32/64 bits | GmAP | Hurt |
Surprising findings
- LSTM remains a strong option in SSVH, performing close to Mamba in the SSM variant ablation.
Limitations
- Bidirectional design doubles per-layer cost, so the model uses 6 encoder layers to keep size comparable to baselines.
- Hash center count and mask ratio are dataset-specific hyperparameters (N_c=450 on FCVID, 100 elsewhere).
Experimental setup
Datasets: ActivityNet FCVID UCF101 HMDB51
Benchmarks: ActivityNet FCVID UCF101 HMDB51
Metrics: mAP@N GmAP precision-recall curves
Baselines: SSVH BTH DKPH MCMSH ConMH BerVAE
Builds on: Mamba S4 BTH ConMH MCMSH
Closest papers by meaning
- Contrastive Masked Autoencoders for Self-Supervised Video Hashing (AAAI 2023 · similarity 0.93)
- AutoSSVH: Exploring Automated Frame Sampling for Efficient Self-Supervised Video Hashing (CVPR 2025 · similarity 0.92)
- Self-Supervised Video Hashing via Bidirectional Transformers (CVPR 2021 · similarity 0.90)
- An Asymmetric Augmented Self-Supervised Learning Method for Unsupervised Fine-Grained Image Hashing (CVPR 2024 · similarity 0.85)
- KALAHash: Knowledge-Anchored Low-Resource Adaptation for Deep Hashing (AAAI 2025 · similarity 0.84)
- Dual Self-Paced Cross-Modal Hashing (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.