Research map / Vision-language and multimodal / CLIP and zero-shot
CLIPCEIL: Domain Generalization through CLIP via Channel rEfinement and Image-text aLignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CLIPCEIL, a lightweight multi-scale Transformer adapter on frozen CLIP ViT-B/16 that refines visual channels for domain invariance and class relevance and realigns image-text directions for multi-source domain generalization.
Problem
Out-of-distribution generalization of CLIP under domain shift, where fine-tuning degrades zero-shot robustness; domain generalization benchmarks require domain-invariant, class-relevant features.
Approach
Penalize per-channel inter-domain variance and reward inter-class variance of the refined visual features, fuse multi-layer CLIP features with a one-layer Transformer adapter, and enforce image-text direction alignment using cross-domain, cross-class pairs; final logits average the adapter and CLIP zero-shot predictions.
Key results
- CLIPCEIL with frozen CLIP ViT-B/16 averages 77.3% across five DG benchmarks, exceeding DPL (75.0%) and StyLIP among frozen-encoder methods.
- CLIPCEIL++ (full encoder fine-tuning) averages 79.1%, versus 78.6% for CLIPood and UniDG.
- On OfficeHome ablation, the full model reaches 85.4% average accuracy versus 82.4% for CLIP zero-shot.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adapt frozen CLIP with channel refinement and image-text alignment | CLIP ViT-B/16 · PACS, VLCS, OfficeHome, TerraIncognita, DomainNet | average accuracy: CLIP Zero-Shot: 70.2; CLIPCEIL: 77.3 ± 0.2 | Helped |
| Fine-tune the full visual encoder using CLIPCEIL++ | CLIP ViT-B/16 · PACS, VLCS, OfficeHome, TerraIncognita, DomainNet | average accuracy: CLIPCEIL++: 79.1 ± 0.2; CLIPood: 78.6 | Helped |
| Add channel refinement loss Lref to multi-scale features (ablation) | CLIPCEIL · OfficeHome | accuracy: 83.2 -> 84.1 | Helped |
| Add direction loss Ldir to multi-scale features (ablation) | CLIPCEIL · OfficeHome | accuracy: 83.2 -> 84.4 | Helped |
| Combine Lref and Ldir with multi-scale features (ablation) | CLIPCEIL · OfficeHome | accuracy: 83.2 -> 85.4 | Helped |
| Use both inter-domain and inter-class variance criteria rather than either alone (ablation) | CLIPCEIL channel refinement · PACS, VLCS, OfficeHome, TerraIncognita, DomainNet | accuracy | Helped |
| Use both multi-scale features and a bypass connection (ablation) | CLIPCEIL adapter · OfficeHome | accuracy: 83.5 -> 85.4 | Helped |
| Replace the Transformer adapter with average pooling (ablation) | CLIPCEIL · OfficeHome | accuracy: 85.4 -> 83.7 | Hurt |
| Replace the Transformer adapter with a two-layer MLP (ablation) | CLIPCEIL · OfficeHome | accuracy: 85.4 -> 84.5 | Hurt |
| Add a multi-scale adapter to the text encoder (ablation) | CLIPCEIL · OfficeHome | accuracy: 85.4 -> 85.0 | Hurt |
Surprising findings
- Simple channel selection on frozen CLIP features (no training) improves OfficeHome average from 82.4% to 83.5%.
Limitations
- Inter-domain variance requires multiple domains, so the method applies only to multi-source DG and not single-source DG.
- Results reported only on ViT-B/16 in main tables; other backbones only evaluated on OfficeHome.
Experimental setup
Datasets: PACS VLCS OfficeHome TerraIncognita DomainNet
Benchmarks: PACS VLCS OfficeHome TerraIncognita DomainNet
Metrics: accuracy
Baselines: CLIP Zero-Shot Linear Probing CoOp CoCoOp CLIP-Adapter MaPLE DPL StyLIP MIRO CLIPood CAR-FT UniDG
Builds on: CLIP CLIPood DomainBed CLIP-Adapter MIRO
Closest papers by meaning
- Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.93)
- When and How Does CLIP Enable Domain and Compositional Generalization? (ICML 2025 · similarity 0.91)
- Unknown Prompt the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization (CVPR 2024 · similarity 0.91)
- Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification (CVPR 2024 · similarity 0.90)
- A Simple Recipe for Language-guided Domain Generalized Segmentation (CVPR 2024 · similarity 0.89)
- OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.