Research map / Vision-language and multimodal / CLIP and zero-shot

Vision-Language Model IP Protection via Prompt-based Learning

CVPR 2025 · Vision-language and multimodal › CLIP and zero-shot · approach: Test-time adaptation of VLMs · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IP-CLIP, a lightweight prompt-learning method that makes a frozen CLIP model work well only on authorized domains and degrade on unauthorized ones, with a style-enhancement feature-bank branch and new weighted IP-protection metrics.

Problem

Protecting the intellectual property of vision-language models like CLIP by restricting deployment to authorized data domains, especially after fine-tuning, without costly retraining; existing IP protection methods rely on visual backbones and extra generated data.

Approach

Learns IP prompts from frozen CLIP visual-encoder style statistics (mean/std) and multi-scale features via an IP projector, producing domain tokens and image tokens fed to the frozen text encoder; adds a style-augment module that uses authorized/unauthorized feature banks and new weighted metrics (Wua, Oua, Dua) to balance authorized and unauthorized performance.

Key results

• On Office-31 target-specified, IP-CLIP reaches mean Wua 74.84% versus 70.09% for CUTI (CNN) and 72.48% for CUTI† (CLIP-based), with Da 0.00%.
• On Office-Home-65 target-specified, mean Wua is 55.10% for IP-CLIP vs 50.29% for CUTI†.
• Ownership verification on Office-31: mean Oua of 71.3% for IP-CLIP vs 65.7% for CUTI† and 52.6% for NTL†.

Measured effects reported in the paper

Surprising findings

• CLIP-based protection methods (NTL†, CUTI†) generally outperform their CNN counterparts, suggesting stronger protection from the CLIP backbone.

Limitations

• Evaluated only on image classification benchmarks (Office-31, Office-Home-65, Mini-DomainNet).
• Target-free setting relies on style augmentation as a proxy for unauthorized domains.
• Metrics and comparisons are based on author-reproduced CLIP adaptations of NTL and CUTI.

Experimental setup

Datasets: Office-31 Office-Home-65 Mini-DomainNet

Benchmarks: Office-31 Office-Home-65 Mini-DomainNet

Metrics: Wua Du Da Oua Dua accuracy

Baselines: NTL CUTI NTL† CUTI† SL-CLIP SL-CNN

Builds on: CoOp MaPLe CLIP NTL CUTI-domain

Closest papers by meaning

• Tracking the Copyright of Large Vision-Language Models through Parameter Learning Adversarial Images (ICLR 2025 · similarity 0.86)
• Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.86)
• FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts (AAAI 2025 · similarity 0.86)
• SIF: Semantically In-Distribution Fingerprints for Large Vision-Language Models (CVPR 2026 · similarity 0.86)
• Backdooring Vision-Language Models with Out-Of-Distribution Data (ICLR 2025 · similarity 0.85)
• BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP (CVPR 2024 · similarity 0.85)