Research map / Video / Video-language understanding
READ-PVLA: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes READ-PVLA, a recurrent bottleneck adapter plus a partial optimal transport video-language alignment loss, to parameter-efficiently fine-tune frozen video-language transformers on low-resource temporal language grounding and video-language summarization.
Problem
Full fine-tuning of large pretrained video-language transformers is storage-costly and unstable with limited data; existing adapters ignore temporal dependencies among frames/words and do not control task-relevant information flowing into the low-dimensional adapter space.
Approach
A recurrent adapter (RNN in a low-dimensional bottleneck) captures temporal dependencies, and a partial optimal transport (POT) alignment loss between video frame and word representations, solved with Sinkhorn, regularizes information entering the adapters.
Key results
- On YouTube Highlights, READ-PVLA reaches 76.12 average mAP with 0.16M trainable parameters, above Full fine-tuning (74.76) and LoRA (69.70).
- On How2 with VG-BART, READ-PVLA obtains 44.01 ROUGE-1 using 1.17M trainable parameters (0.47%), versus 35.72 for Full fine-tuning.
- On QVHighlights, READ-PVLA reaches 36.74 mAP versus 36.14 for Full fine-tuning, with 0.19M parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add READ-PVLA instead of full fine-tuning | UMT · YouTube Highlights | mAP: 74.76‡ -> 76.12 | Helped |
| add READ-PVLA instead of full fine-tuning | UMT · TVSum | mAP: 79.65‡ -> 83.39 | Helped |
| add READ-PVLA instead of full fine-tuning | Moment-DETR · QVHighlights | mAP: 36.14‡ -> 36.74 | Helped |
| add READ-PVLA instead of Adapter | VG-BART · How2 | ROUGE-1: 41.52‡ -> 44.01 | Helped |
| add READ-PVLA instead of Adapter | VG-BART · How2 | ROUGE-2: 20.75‡ -> 21.91 | Helped |
| add READ-PVLA instead of Adapter | VG-BART · How2 | ROUGE-L: 36.88‡ -> 37.91 | Helped |
| add READ-PVLA instead of Adapter | VG-T5 · How2 | ROUGE-1: 37.83‡ -> 40.12 | Helped |
| add READ-PVLA instead of Adapter | VG-T5 · How2 | ROUGE-2: 17.45‡ -> 18.71 | Helped |
| add READ-PVLA instead of Adapter | VG-T5 · How2 | ROUGE-L: 33.87‡ -> 34.42 | Helped |
| add partial video-language alignment rather than removing alignment (ablation) | READ · YouTube Highlights | mAP: 73.80 -> 76.12 | Helped |
| use partial alignment rather than full-mass VLA (ablation) | READ · YouTube Highlights | mAP: 74.41 -> 76.12 | Helped |
| replace RNN with GRU or LSTM (ablation) | READ · YouTube Highlights and How2 | mAP / ROUGE-2: 76.07, 76.08, 76.12; 21.85, 21.87, 21.91 | No clear effect |
Surprising findings
- Replacing the partial OT objective with a full-mass variant (VLA) slightly decreased performance relative to PVLA on YouTube Highlights and How2.
- Recurrent cell choice (RNN, GRU, LSTM) made little difference in performance.
Limitations
- Low-resource experiments use subsampled training sets (e.g., 700 of QVHighlights, 2,000 of How2), so gains at full data scale are not shown.
- PVLA requires Sinkhorn iterations (up to 1,000) per loss computation, adding training cost not analyzed in the text.
Experimental setup
Datasets: YouTube Highlights TVSum QVHighlights How2
Benchmarks: YouTube Highlights TVSum QVHighlights How2
Metrics: mAP ROUGE-1 ROUGE-2 ROUGE-L
Baselines: Full fine-tuning Partial Bias Proj LoRA Prompt Adapter
Builds on: Adapter (Houlsby et al. 2019) LoRA Moment-DETR UMT VG-BART
Closest papers by meaning
- VidLA: Video-Language Alignment at Scale (CVPR 2024 · similarity 0.88)
- Efficient Transfer Learning for Video-language Foundation Models (CVPR 2025 · similarity 0.88)
- PAVE: Patching and Adapting Video Large Language Models (CVPR 2025 · similarity 0.87)
- Time-R1: Post-Training Large Vision Language Model for Temporal Video Grounding (NeurIPS 2025 · similarity 0.87)
- Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization (NeurIPS 2025 · similarity 0.87)
- p-Laplacian Adaptation for Generative Pre-trained Vision-Language Models (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.