Research map / Architectures and efficiency / Pruning and efficient tuning
COBRA: Contribution-Based Bayesian Rank Allocation for Parameter-Efficient Fine-Tuning
Hongcheng Ding, Xuanze Zhao, Xuanhuang Liu, Jing Jin, Shamsul Nahar Abdullah, Deshinta Arrova Dewi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace uniform rank (LoRA r=8) with COBRA heterogeneous ranks | LoRA · GLUE avg, RoBERTa-base | Avg. score: 87.0 -> 88.6 | Improved |
| use COBRA instead of AdaLoRA | AdaLoRA · GLUE avg, DeBERTaV3-base | Avg. score: 89.0 -> 89.9 | Improved |
| use COBRA instead of gradient-driven GoRA | GoRA · T5-Base GLUE-5 | Avg. score: 88.0 -> 89.1 | Improved |
| use COBRA instead of GoRA | GoRA · LLaMA-3.1-8B HumanEval | Accuracy: 49.0 -> 52.7 | Improved |
| use COBRA instead of uniform LoRA in high-rank regime | LoRA r=384 · NEU financial sentiment | MSE: 0.0189 -> 0.0177 | Improved |
| use multiplicative (Phi x Gamma) instead of additive aggregation (ablation) | COBRA · GLUE avg, DeBERTaV3-base | Avg. score: 89.2 -> 89.9 | Improved |
| combine contribution and adaptation demand instead of a single factor (ablation) | COBRA · GLUE avg, DeBERTaV3-base | Avg. score: 88.8 -> 89.9 | Improved |
| use informative TA-LC prior (gamma=5) instead of uniform prior (gamma=0) (ablation) | COBRA · GLUE avg, DeBERTaV3-base | Avg. score: 88.5 -> 89.9 | Improved |
| increase prior concentration to gamma=50 (ablation) | COBRA · GLUE avg, DeBERTaV3-base | Avg. score: 89.9 -> 88.3 | Hurt |
| add KL regularization (beta=0.01) vs none (ablation) | COBRA · GLUE avg, DeBERTaV3-base | Avg. score: 88.8 -> 89.9 | Improved |
| use strong KL weight beta=1.0 (ablation) | COBRA · GLUE avg, DeBERTaV3-base | Avg. score: 89.9 -> 88.2 | Hurt |
| use DyLoRA range-based dynamic allocation instead of uniform LoRA | LoRA · DeBERTaV3-base GLUE r=4 | Avg. score: 88.7 -> 88.4 | Hurt |
Closest papers by meaning
- GoRA: Gradient-driven Adaptive Low Rank Adaptation (NeurIPS 2025 · similarity 0.90)
- Beyond LoRA vs. Full Fine-Tuning: Gradient-Guided Optimizer Routing for LLM Adaptation (NeurIPS 2026 · similarity 0.89)
- Unveiling LoRA Intrinsic Ranks via Salience Analysis (NeurIPS 2024 · similarity 0.89)
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation (NeurIPS 2025 · similarity 0.89)
- LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning (NeurIPS 2024 · similarity 0.88)
- Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
