Research map / Label-efficient and robust learning / Domain adaptation and generalization

PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization

CVPR 2024 · Label-efficient and robust learning › Domain adaptation and generalization · approach: Foundation-model domain generalization · 16 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SCI-PD, a perturbation distillation method that transfers CLIP's zero-shot knowledge to lightweight vision models for open-set domain generalization, and introduces the HDG benchmark with the H2-CV robustness metric.

Problem

Domain generalization methods assume identical label sets across source domains and fixed splits, and open-set DG methods are computationally costly with little gain; VLM-based DG fine-tunes large models, which is expensive.

Approach

Distill CLIP's similarity distribution into lightweight vision models (e.g., ResNet18) using three perturbations: score perturbation injects GT labels into CLIP similarity scores, instance perturbation reweights samples by inverse max similarity, and class perturbation aligns classifier weights with CLIP text embedding similarities. Also proposes a hybridness-controlled benchmark (HDG) with varied

Key results

• On OfficeHome, SCI-PD reaches 61.73% average accuracy and 57.33% average H-score with H2-CV 5.20%, versus 58.11%/53.92% and H2-CV 6.40% for CLIPBase.
• On PACS, SCI-PD achieves 70.16% average accuracy and 61.21% H-score, with H2-CV 21.27% versus 23.15% for CLIPBase.
• On DomainNet, SCI-PD improves average accuracy to 34.16% over CLIPBase's 31.43%, although its H2-CV (13.64%) is higher than CLIPBase's 11.50%.

Measured effects reported in the paper

Surprising findings

• On DomainNet, SCI-PD has a higher H2-CV (13.64%) than CLIPBase (11.50%), so robustness is not uniformly improved there.

Limitations

• Gains on DomainNet are modest and H2-CV is worse than CLIPBase there.
• Hybridness splits are preset to four values, so robustness is assessed on a limited set of splits.

Experimental setup

Datasets: PACS OfficeHome DomainNet

Benchmarks: PACS OfficeHome DomainNet

Metrics: Top-1 accuracy H-score H2-CV

Baselines: ERM CORAL MMD RSC MixStyle CIRL XDED RISE ARPL DAML EDir-CORAL MEDIC

Builds on: CLIP ERM MIRO DAML MEDIC

Closest papers by meaning

• Leveraging Vision-Language Models for Improving Domain Generalization in Image Classification (CVPR 2024 · similarity 0.89)
• Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap (ICLR 2025 · similarity 0.89)
• Domain Generalization in CLIP via Learning with Diverse Text Prompts (CVPR 2025 · similarity 0.89)
• Is Large-scale Pretraining the Secret to Good Domain Generalization? (ICLR 2025 · similarity 0.89)
• AdvST: Revisiting Data Augmentations for Single Domain Generalization (AAAI 2024 · similarity 0.89)
• Unknown Domain Inconsistency Minimization for Domain Generalization (ICLR 2024 · similarity 0.88)