Research map / Vision-language and multimodal / CLIP and zero-shot
Retrieval Across Any Domains via Large-scale Pre-trained Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TKI, a data-free method that fine-tunes a text-driven non-Euclidean MLP projection on CLIP using learnable domain word vectors, enabling cross-domain image retrieval without any training images.
Problem
Cross-domain image retrieval generalization to unseen domains normally needs costly multi-domain training data; the paper targets data-free adaptive cross-domain retrieval with no real images during training.
Approach
Insert learnable domain word vectors into CLIP text prompts ("aggregation after expansion"), then train a hyperbolic (Poincaré-ball) MLP projection with supervised contrastive loss on text features only; at test time, the projection is stacked on CLIP's visual encoder, exploiting cross-modal transferability between text and image features.
Key results
- On PACS with ViT-B/32, TKI averages 73.12 mAP vs. 61.09 for vanilla CLIP (Table 5/3 reporting).
- On Office-Home, TKI averages 37.11 mAP vs. 30.17 for vanilla CLIP for the A-source setting.
- On DomainNet, average mAP rises from 15.76 (vanilla CLIP) to 21.31 (TKI) without any training images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · DomainNet, Real query (Avg.) | mAP: 15.76 -> 21.31 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · DomainNet, Sketch query (Avg.) | mAP: 11.34 -> 16.75 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · DomainNet, Quickdraw query (Avg.) | mAP: 0.80 -> 1.97 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · DomainNet, Infograph query (Avg.) | mAP: 10.22 -> 13.15 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · DomainNet, Painting query (Avg.) | mAP: 13.77 -> 18.55 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · DomainNet, Clip-art query (Avg.) | mAP: 12.25 -> 18.33 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · PACS, Art Painting query (Avg.) | mAP: 61.09 -> 73.12 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · PACS, Cartoon query (Avg.) | mAP: 58.51 -> 74.25 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · PACS, Photo query (Avg.) | mAP: 71.70 -> 84.80 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · PACS, Sketch query (Avg.) | mAP: 49.36 -> 68.53 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · Office-Home, Art query (Avg.) | mAP: 30.17 -> 37.11 | Helped |
| Apply TKI instead of using the unadapted visual encoder | CLIP ViT-B/32 · Office-Home, Clipart query (Avg.) | mAP: 20.45 -> 27.95 | Helped |
Surprising findings
- Randomly chosen text labels that do not overlap with the dataset's classes still improve over the baseline, suggesting zero-shot generalization of the learned projection.
Limitations
- Only evaluated on three benchmarks with CLIP backbones; generalization to other pre-trained models is untested.
- Performance on DomainNet remains low in absolute terms (avg mAP about 21%).
Experimental setup
Datasets: DomainNet PACS Office-Home
Benchmarks: DomainNet PACS Office-Home
Metrics: mAP mAP@200 Prec@200
Baselines: Vanilla CLIP
Builds on: CLIP universal cross-domain retrieval
Closest papers by meaning
- Semantic Feature Learning for Universal Unsupervised Cross-Domain Retrieval (NeurIPS 2024 · similarity 0.88)
- Adaptive Prompt-Based Semantic Embedding with Inspire Potential of Implicit Knowledge for Cross-Modal Retrieval (AAAI 2025 · similarity 0.87)
- Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation (CVPR 2024 · similarity 0.86)
- Domain-Controlled Prompt Learning (AAAI 2024 · similarity 0.86)
- Target Semantics Clustering via Text Representations for Robust Universal Domain Adaptation (AAAI 2025 · similarity 0.86)
- Unsupervised Cross-Domain Image Retrieval via Prototypical Optimal Transport (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.