Research map / Architectures and efficiency / Pruning and efficient tuning
Learn to Preserve and Diversify: Parameter-Efficient Group with Orthogonal Regularization for Domain Generalization
Jiajun Hu, Jian Zhang, Lei Qi, Yinghuan Shi, Yang Gao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add orthogonal regularization and a LoRA group (PEGO) on top of LoRA | LoRA on CLIP ViT-B/16 · DomainBed 5-dataset average | accuracy: 75.0 -> 76.1 | Improved |
| add L_diversify to L_preserve (ablation) | PEGO · OfficeHome | accuracy: 82.95 -> 84.21 | Improved |
| add L_preserve to L_diversify (ablation) | PEGO · PACS | accuracy: 96.37 -> 96.55 | Improved |
| add both orthogonal losses to an unregularized LoRA group (ablation) | LoRA group · PACS | accuracy: 95.34 -> 96.55 | Improved |
| use a group of LoRA modules without regularization instead of single LoRA (ablation) | LoRA · PACS and OfficeHome | accuracy: 95.96 -> 95.34 (PACS); 83.41 -> 82.85 (OH) | Hurt |
| use PEFT (LoRA, VPT, Adapter) instead of full fine-tuning | CLIP ViT-B/16 · DomainBed 5-dataset average | accuracy: 61.1 -> 75.0 (LoRA) | Improved |
| increase LoRA rank above 8 (ablation) | Baseline LoRA group vs PEGO · PACS and OfficeHome | accuracy | Mixed |
| vary balancing coefficient from 1e-4 to 1e-1 (ablation) | PEGO · PACS and OfficeHome | accuracy | Improved |
| PEGO versus zero-shot CLIP on TerraIncognita | CLIP ViT-B/16 · TerraIncognita | accuracy: 31.3 -> 57.3 | Improved |
| PEGO versus zero-shot CLIP on PACS | CLIP ViT-B/16 · PACS | accuracy: 96.8 -> 96.5 | Hurt |
| scale backbone to ViT-L/14 and apply PEGO instead of LoRA | LoRA on CLIP ViT-L/14 · TerraIncognita | accuracy: 52.7 -> 57.2 | Improved |
| PEGO versus LoRA on PACS and VLCS with ViT-L/14 | CLIP ViT-L/14 · PACS, VLCS | accuracy: 98.1 vs 98.0 (PACS) | No clear effect |
Closest papers by meaning
- PracticalDG: Perturbation Distillation on Vision-Language Models for Hybrid Domain Generalization (CVPR 2024 · similarity 0.90)
- Domain Generalization: A Survey (TPAMI 2022 · similarity 0.89)
- Gradient-Guided Annealing for Domain Generalization (CVPR 2025 · similarity 0.89)
- PACE: Marrying generalization in PArameter-efficient fine-tuning with Consistency rEgularization (NeurIPS 2024 · similarity 0.88)
- Seeking Consistent Flat Minima for Better Domain Generalization via Refining Loss Landscapes (CVPR 2025 · similarity 0.88)
- Sharpness-Aware Model-Agnostic Long-Tailed Domain Generalization (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
