Research map / Label-efficient and robust learning / Domain adaptation and generalization
Unknown Prompt the only Lacuna: Unveiling CLIP's Potential for Open Domain Generalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ODG-CLIP, which casts open domain generalization as (C+1)-way classification with CLIP prompt learning, using Stable Diffusion to synthesize pseudo-unknown images and a style-conditioned prompt plus a consistency loss to enrich CLIP visual embeddings.
Problem
Open Domain Generalization (ODG) requires classifying known categories and detecting unknown-category samples in unlabeled target domains that shift in both domain and category; prior CNN-based ODG methods generalize poorly and CLIP prompt methods lack outlier handling.
Approach
Model an unknown-class prompt trained on Stable Diffusion-generated pseudo-open images (positive/negative prompts), use a domain token derived from CLIP visual feature statistics with generic learnable tokens, and enhance visual embeddings by subtracting class-conditioned from domain-only prompt embeddings, enforced by a cross-domain semantic consistency loss.
Key results
- ODG-CLIP reaches an average H-score of 90.84% and accuracy of 94.23% across six ODG benchmarks, versus 78.64% H-score for the best competitor (STYLIP + SD).
- On closed-set DG, ODG-CLIP averages 97.10% vs 86.67% for STYLIP across PACS, VLCS, Office-Home, Digits-DG and Mini-DomainNet.
- Diffusion-generated pseudo-open images achieve an inception score of 6.67 vs 1.18 for Cumix and 1.43 for OpenGAN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ODG-CLIP instead of the strongest reported baseline | ODG methods · PACS, VLCS, Office-Home, Digits-DG, Multi-Dataset, Mini-DomainNet | H-score: 99.70 vs 87.95; 86.53 vs 68.02; 96.08 vs 83.31; 78.27 vs 68.10; 90.00 vs 78.52; 94.48 vs 82.95 | Helped |
| Use ODG-CLIP instead of the strongest reported prompt-learning baseline | closed-set DG · PACS, VLCS, Office-Home, Digits-DG, Mini-DomainNet | accuracy: 99.83 vs 98.17; 95.74 vs 87.25; 96.91 vs 85.94; 96.38 vs 82.40; 96.65 vs 80.43 | Helped |
| Add Lsem (ablation) | ODG-CLIP · PACS | H-score: 92.56 -> 99.70 | Helped |
| Add latent visual information to the image and use Lsem (ablation) | ODG-CLIP · PACS | H-score: 88.34 -> 99.70 | Helped |
| Derive latent visual information from domain-conditioned learned prompt differences rather than fixed class-name embeddings (ablation) | ODG-CLIP · PACS | H-score: 92.82 -> 99.70 | Helped |
| Generate pseudo-open images with stable diffusion instead of OpenGAN (ablation) | ODG-CLIP · PACS | H-score: 91.35 -> 99.70 | Helped |
| Generate pseudo-open images with stable diffusion instead of Cumix (ablation) | ODG-CLIP · PACS | H-score: 93.08 -> 99.70 | Helped |
| Use the proposed domain-aware prompts rather than manually defined prompts for both prompt types (ablation) | ODG-CLIP · PACS | H-score: 80.56 -> 99.70 | Helped |
| Learn generic prompt tokens instead of manually specifying them (ablation) | ODG-CLIP · PACS | H-score: 84.61 -> 99.70 | Helped |
| Initialize generic prompt tokens from the embedding of “Image of a” rather than Gaussian noise (ablation) | ODG-CLIP · PACS | H-score: 93.84 -> 99.70 | Helped |
| Use positive and negative prompts for stable-diffusion pseudo-open image generation instead of positive prompts alone (ablation) | ODG-CLIP · PACS | H-score: 92.16 -> 99.70 | Helped |
| Use positive and negative prompts for stable-diffusion pseudo-open image generation instead of positive prompts alone (ablation) | ODG-CLIP · Office-Home | H-score: 90.83 -> 96.08 | Helped |
Surprising findings
- Removing the cross-domain consistency loss while keeping the visual enhancement still gives strong results, but a manually derived class-name embedding for the enhancement drops H-score by 5-11 points.
- Simply adding an unknown-class prompt trained with SD images to prior CLIP prompt methods (MaPLe, PromptSRC, STYLIP) boosts their H-score substantially, indicating the pseudo-open data is a key ingredient.
Limitations
- Relies on pseudo-open images from a diffusion model whose quality and semantics are uncontrolled; a entropy threshold filter is heuristic.
- Evaluated only on classification benchmarks; dense prediction tasks left for future work.
- Results are averaged over three runs with no compute or hardware details reported.
Experimental setup
Datasets: PACS VLCS Office-Home Digits-DG Multi-Dataset Mini-DomainNet
Benchmarks: PACS VLCS Office-Home Digits-DG Multi-Dataset Mini-DomainNet
Metrics: top-1 accuracy H-score inception score
Baselines: CLIP CLIP + OpenMax CLIP + OSDA CoOp CoCoOp MaPLe LASP PromptSRC CLIPN STYLIP DAML MEDIC
Builds on: CLIP StyLIP CoOp CLIPN Stable Diffusion v1-5
Closest papers by meaning
- OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP (CVPR 2025 · similarity 0.94)
- Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.92)
- CLIPCEIL: Domain Generalization through CLIP via Channel rEfinement and Image-text aLignment (NeurIPS 2024 · similarity 0.91)
- When and How Does CLIP Enable Domain and Compositional Generalization? (ICML 2025 · similarity 0.89)
- PromptKD: Unsupervised Prompt Distillation for Vision-Language Models (CVPR 2024 · similarity 0.89)
- Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.