Research map / Vision-language and multimodal / CLIP and zero-shot
HALoRA: Low-Rank Adaptation with Hierarchical Budget Allocation for Efficient Vision-Language Alignment
Letian Zhang, GuangHao Meng, XuDong Ren, Jinpeng Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use component-level instead of model-level rank allocator (ablation) | HALoRA · Flickr30k retrieval | TR@1: 55.6 -> 56.8 | Improved |
| use component-level instead of modality-level allocator (ablation) | HALoRA · Flickr30k retrieval | TR@1: 54.9 -> 56.8 | Improved |
| use component-level instead of model-level allocator (ablation) | HALoRA · UCF101 zero-shot | Acc-1: 26.4 -> 27.2 | Improved |
| add gradient-approximation initialization (ablation) | HALoRA · Flickr30k retrieval | TR (mean of R@1,5,10): 74.2 -> 76.4 | Improved |
| add gradient-approximation initialization (ablation) | HALoRA · ImageNet-A zero-shot | Acc-1: 6.69 -> 6.54 | Hurt |
| add gradient-approximation initialization (ablation) | HALoRA · UCF101 zero-shot | Acc-1: 24.8 -> 27.3 | Improved |
| use asymmetric instead of symmetric manual ranks (ablation) | LoRA-GA · Flickr30k retrieval | TR@1: 53.7 -> 56.6 | Improved |
| use asymmetric instead of uniform ranks (ablation) | LoRA-GA · Flickr30k retrieval | IR@1: 42.0 -> 42.4 | Improved |
| use asymmetric ranks instead of uniform (ablation) | CLIP-LoRA · 11-dataset few-shot (1-shot, avg) | accuracy: 72.5 -> 72.7 | Improved |
| use HALoRA instead of AdaLoRA | LoRA-based PEFT · Flickr30k retrieval | TR@1: 52.8 -> 56.8 | Improved |
| use plain LoRA instead of adapters (LilT LwA) | dual-encoder PEFT · Flickr30k retrieval | IR@1: 41.7 -> 40.9 | Hurt |
Closest papers by meaning
- MARS: Harmonizing Multimodal Convergence via Adaptive Rank Search (NeurIPS 2026 · similarity 0.90)
- Exploring the Better Multimodal Synergy Strategy for Vision-Language Models (AAAI 2025 · similarity 0.90)
- Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models (NeurIPS 2025 · similarity 0.89)
- LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models (CVPR 2025 · similarity 0.89)
- IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities (AAAI 2025 · similarity 0.88)
- LoMo: Local Modality Substitution for Deeper Vision-Language Fusion (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
