Research map / Video / Video-language understanding
Federated Weakly Supervised Video Anomaly Detection with Multimodal Prompt
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a federated weakly supervised video anomaly detector built on a frozen CLIP, using a text prompt generator conditioned on global text class contexts and local averaged visual contexts.
Problem
Weakly supervised video anomaly detection needs abnormal videos that institutions cannot share due to privacy, so training a generalized detector across clients without centralizing data is difficult.
Approach
A prompt generator (cross-attention) produces CLIP text prompts where queries come from client-local averaged visual features and keys/values come from global anomaly class text embeddings, balancing global generalization and local personalization in federated learning.
Key results
- Under federated training with random/event/scene splits, the method reaches 84.06/85.07/84.03 AUC on UCF-Crime and 75.32/74.23/75.99 AP on XD-Violence, outperforming CLAP (e.g., 68.60/66.57/70.54 AP on XD-Violence).
- Compared with local training, federated training gains 2.89% AUC on UCF-Crime and 10.44% AP on XD-Violence.
- Trained on XD-Violence, the method reaches 54.62% AUC on ShanghaiTech, outperforming ZS-CLIP and Temporal-CLIP on average by 3.83% and 9.32%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace global text context with local text context (ablation) | Global and Local Context-driven Federated Learning · UCF-Crime, average across data splits | AUC (%): 84.10 vs 83.62 | Hurt |
| replace global text context with local text context (ablation) | Global and Local Context-driven Federated Learning · XD-Violence, average across data splits | AP (%): 75.18 vs 71.58 | Hurt |
| replace local visual context with learnable vectors (ablation) | Global and Local Context-driven Federated Learning · XD-Violence, average across data splits | AP (%): 75.18 vs 73.29 | Hurt |
| use federated rather than local training | Ours · UCF-Crime | AUC (%): +2.89% | Helped |
| use federated rather than local training | Ours · XD-Violence | AP (%): +10.44% | Helped |
| train on XD-Violence and evaluate on ShanghaiTech | Ours · ShanghaiTech test | AUC (%): 54.62% AUC | Helped |
Surprising findings
- Centralized training with the proposed method is not better than the baseline on the UCF-Crime dataset, which the authors attribute to the design targeting federated balance.
Limitations
- Only evaluated with CLIP ViT-B/16 features and a single federated setup of clients.
- Gains are modest on some splits (e.g., 84.06 vs 84.99 on UCF-Crime random split versus CLAP).
- Centralized performance is not superior on all datasets.
Experimental setup
Datasets: UCF-Crime XD-Violence ShanghaiTech
Benchmarks: UCF-Crime XD-Violence ShanghaiTech
Metrics: AUC AP
Baselines: ZS-CLIP Temporal-CLIP FedCoOp PPVU CLAP
Builds on: CLIP CoOp FedTPG FedAvg CLAP
Closest papers by meaning
- Prompt-Enhanced Multiple Instance Learning for Weakly Supervised Video Anomaly Detection (CVPR 2024 · similarity 0.91)
- Learning Event Completeness for Weakly Supervised Video Anomaly Detection (ICML 2025 · similarity 0.89)
- VadCLIP: Adapting Vision-Language Models for Weakly Supervised Video Anomaly Detection (AAAI 2024 · similarity 0.89)
- Text Prompt with Normality Guidance for Weakly Supervised Video Anomaly Detection (CVPR 2024 · similarity 0.88)
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.88)
- VADTree: Explainable Training-Free Video Anomaly Detection via Hierarchical Granularity-Aware Tree (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.