Research map / Vision-language and multimodal / CLIP and zero-shot
In-context Prompt-augmented Micro-video Popularity Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICPF, a model-agnostic in-context prompt-augmented framework that retrieves similar micro-videos and uses their features as learnable prompts to prompt-tune frozen multimodal transformers (CLIP) for micro-video popularity prediction.
Problem
Micro-video popularity prediction suffers from limited contextual information and incomplete modal semantics, while full fine-tuning of pre-trained multimodal transformers is inefficient and unstable on small downstream datasets.
Approach
A multi-branch retriever finds similar videos via within-modality similarity; a cross-attention in-context prompt generator turns retrieved text/visual features (and popularity labels) into dynamic prompts that are inserted into frozen CLIP-style encoders, with only the prompt generator and a knowledge-augmented predictor trained.
Key results
- ICPF improves over the best baselines on TikTok by 22.30% in nMSE, 9.53% in MAE and 8.04% in SRC (0.3048 nMSE, 1.2539 MAE, 0.7554 SRC).
- ICPF reports nMSE 0.7338 on MicroLens and 0.5730 on NUS, outperforming all 14 baselines across three datasets.
- Adding in-context prompts to CLIP, BLIP, and two other backbones improves MAE/SRC, e.g. BLIP on MicroLens SRC rises from 0.2731 to 0.4949.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ICPF instead of the strongest baseline, CLIP* | MicroLens · MicroLens | nMSE: 0.7711 -> 0.7338 | Helped |
| use ICPF instead of the strongest baseline, CLIP* | MicroLens · MicroLens | MAE: 1.1867 -> 1.1460 | Helped |
| use ICPF instead of the strongest baseline, CLIP* | MicroLens · MicroLens | SRC: 0.4795 -> 0.5200 | Helped |
| use ICPF instead of the strongest baseline, CLIP* | NUS · NUS | nMSE: 0.5978 -> 0.5730 | Helped |
| use ICPF instead of the strongest baseline, CLIP* | NUS · NUS | MAE: 1.5665 -> 1.5199 | Helped |
| use ICPF instead of the strongest baseline, CLIP* | NUS · NUS | SRC: 0.6278 -> 0.6548 | Helped |
| use ICPF instead of the strongest baseline, CBAN | TikTok · TikTok | nMSE: 0.3923 -> 0.3048 | Helped |
| use ICPF instead of the strongest baseline, MMRA | TikTok · TikTok | MAE: 1.3861 -> 1.2539 | Helped |
| use ICPF instead of the strongest baseline, CBAN | TikTok · TikTok | SRC: 0.6992 -> 0.7554 | Helped |
| use multi-branch retrieval rather than cross-modal retrieval (ablation) | ICPF · MicroLens | nMSE: 0.7441 -> 0.7338 | Helped |
| use multi-branch retrieval rather than cross-modal retrieval (ablation) | ICPF · TikTok | nMSE: 0.3052 -> 0.3048 | Helped |
| use a retriever rather than random instances (ablation) | ICPF · TikTok | nMSE: 0.3180 -> 0.3048 | Helped |
Surprising findings
- Static prompt tuning (MaPLe, CLIP*, BLIP*) performs worse than many feature-engineering and fusion baselines, which the authors attribute to instance-agnostic prompts.
- Cross-modal retrieval performs worse than within-modality retrieval, contrary to the common assumption that cross-modal matching is most suitable.
Limitations
- Performance on TikTok is sensitive to the number of retrieved instances K and prompt length L, with degradation at larger values attributed to noise.
- Evaluated only on popularity regression over three datasets with random 8:1:1 splits.
Experimental setup
Datasets: MicroLens NUS TikTok
Benchmarks: MicroLens NUS TikTok
Metrics: nMSE MAE SRC
Baselines: SVR HyFea MFTM CLSTM TMALL MASSL CBAN HMMVE JAB MQMC MMRA CLIP
Builds on: CLIP CoOp VPT MaPLe MMRA
Closest papers by meaning
- Improving Multimodal Social Media Popularity Prediction via Selective Retrieval Knowledge Augmentation (AAAI 2025 · similarity 0.86)
- SMTPD: A New Benchmark for Temporal Prediction of Social Media Popularity (CVPR 2025 · similarity 0.83)
- Retrieval-Augmented Dynamic Prompt Tuning for Incomplete Multimodal Learning (AAAI 2025 · similarity 0.83)
- An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability (ICML 2025 · similarity 0.83)
- Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis (CVPR 2025 · similarity 0.82)
- Multimodal Prompt Optimization: Why Not Leverage Multiple Modalities for MLLMs (ICLR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.