Research map / Video / Video-language understanding
VarCMP: Adapting Cross-Modal Pre-Training Models for Video Anomaly Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VarCMP, which adapts frozen CLIP and CLAP encoders with a unified multi-granularity alignment and anomaly-biased weighting for video anomaly retrieval on UCFCrime-AR and XDViolence-AR.
Problem
Video anomaly retrieval: retrieving abnormal or normal long, untrimmed videos given text or audio queries, where anomalies occupy only parts of videos and existing methods do not exploit strong cross-modal pre-trained models.
Approach
Uses frozen CLIP (image/text) and CLAP (audio) encoders with a unified hierarchical (video-level, frame-level, patch-level) alignment for video-text and video-audio matching, plus an anomaly-biased weighting that uses a weakly supervised anomaly classifier to up-weight anomalous frames and patches in fine-grained alignment.
Key results
- On UCFCrime-AR text-to-video, VarCMP reaches 28.6 R@1, a 5.0-point gain over UCoFiA (23.6).
- On XDViolence-AR audio-to-video, VarCMP reaches 38.2 R@1, a 5.3-point gain over ALAN† (32.9).
- Removing anomaly-biased weighting drops XDViolence-AR audio-to-video R@1 from 38.2 to 35.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VarCMP instead of UCoFiA | UCFCrime-AR text-to-video retrieval · UCFCrime-AR, Text → Video | R@1: 23.6 -> 28.6 | Helped |
| Use VarCMP instead of UCoFiA | UCFCrime-AR video-to-text retrieval · UCFCrime-AR, Video → Text | R@1: 22.8 -> 24.1 | Helped |
| Use VarCMP instead of ALAN† | XDViolence-AR audio-to-video retrieval · XDViolence-AR, Audio → Video | R@1: 32.9 -> 38.2 | Helped |
| Use VarCMP instead of ALAN† | XDViolence-AR video-to-audio retrieval · XDViolence-AR, Video → Audio | R@1: 34.3 -> 38.5 | Helped |
| Add fine alignment to coarse alignment (ablation) | VarCMP on UCFCrime-AR text-to-video retrieval · UCFCrime-AR, Text → Video | R@1: 24.7 -> 26.0 | Helped |
| Add finer-grained alignment to coarse and fine alignment (ablation) | VarCMP on UCFCrime-AR text-to-video retrieval · UCFCrime-AR, Text → Video | R@1: 26.0 -> 28.6 | Helped |
| Add fine alignment to coarse alignment (ablation) | VarCMP on XDViolence-AR audio-to-video retrieval · XDViolence-AR, Audio → Video | R@1: 31.4 -> 34.0 | Helped |
| Add finer-grained alignment to coarse and fine alignment (ablation) | VarCMP on XDViolence-AR audio-to-video retrieval · XDViolence-AR, Audio → Video | R@1: 34.0 -> 38.2 | Helped |
| Add anomaly-biased weighting (ablation) | VarCMP on UCFCrime-AR text-to-video retrieval · UCFCrime-AR, Text → Video | R@1: 27.9 -> 28.6 | Helped |
| Add anomaly-biased weighting (ablation) | VarCMP on UCFCrime-AR video-to-text retrieval · UCFCrime-AR, Video → Text | R@1: 25.9 -> 24.1 | Hurt |
| Add anomaly-biased weighting (ablation) | VarCMP on XDViolence-AR audio-to-video retrieval · XDViolence-AR, Audio → Video | R@1: 35.9 -> 38.2 | Helped |
| Add anomaly-biased weighting (ablation) | VarCMP on XDViolence-AR video-to-audio retrieval · XDViolence-AR, Video → Audio | R@1: 34.4 -> 38.5 | Helped |
Limitations
- Anomaly-biased weighting relies on anomaly labels during training to decide which videos get anomaly weights.
- Encoders are kept frozen, so adaptation is limited to the added alignment and weighting modules.
Experimental setup
Datasets: UCFCrime-AR XDViolence-AR
Benchmarks: UCFCrime-AR XDViolence-AR
Metrics: R@1 R@5 R@10 MdR
Baselines: CE MMT HL-Net XML T2VLAD X-CLIP ALAN CLIP4Clip TS2-Net UCoFiA
Builds on: CLIP4Clip UCoFiA VadCLIP CLAP ALAN
Closest papers by meaning
- Holmes-VAU: Towards Long-term Video Anomaly Understanding at Any Granularity (CVPR 2025 · similarity 0.86)
- Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval (AAAI 2024 · similarity 0.85)
- MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence (NeurIPS 2025 · similarity 0.85)
- Beyond Single-Modal Boundary: Cross-Modal Anomaly Detection through Visual Prototype and Harmonization (CVPR 2025 · similarity 0.85)
- Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension (NeurIPS 2025 · similarity 0.85)
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.