Research map / Architectures and efficiency / Pruning and efficient tuning
Adaptive Utilization of Low-Rank Adaptation via Conditioned Gating
Guang Yang, Changhao Guan, Chao Huang, Yufeng Chen, Kaiyu Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add token-wise utilization gating (ablation) | U-LoRA · Math reasoning avg | avg accuracy: 78.95 -> 83.97 | Improved |
| add sequence-level aggregation (ablation) | U-LoRA · Math reasoning avg | avg accuracy: 79.16 -> 83.97 | Improved |
| add bias-corrected EMA historical prior (ablation) | U-LoRA · Math reasoning avg | avg accuracy: 79.66 -> 83.97 | Improved |
| add sequence-level aggregation (ablation) | U-LoRA · AQuA | accuracy: 31.89 -> 43.31 | Improved |
| use U-LoRA instead of increasing LoRA rank to 32 | LLaMA-3-8B · Math reasoning avg | avg accuracy: 71.67 -> 76.02 | Improved |
| increase LoRA rank from 16 to 32 (ablation) | LoRA · GLUE avg (RoBERTa-Base) | avg score: 82.82 -> 83.60 | Improved |
| U-LoRA versus LoRA across ranks 2 to 32 (ablation) | Qwen3-8B · Math reasoning avg | avg accuracy: 77.60 -> 82.06 (r=2) | Improved |
| apply U-LoRA to more target modules (Q to QKVOGUD) (ablation) | Qwen3-8B · Math reasoning avg | avg accuracy: 80.75 -> 84.20 | Improved |
| vary EMA momentum beta (0.9 to 0.9995) (ablation) | U-LoRA · Math reasoning avg | avg accuracy: 83.23 -> 84.17 (0.9 vs 0.995) | No clear effect |
| U-LoRA instead of TopLoRA at 72B scale | Qwen2.5-72B Base · Math reasoning avg | avg accuracy: 83.97 -> 85.08 | Improved |
| U-LoRA instead of TopLoRA on instruct model | Qwen2.5-14B-Instruct · Math reasoning avg | avg accuracy: 79.58 -> 82.82 | Improved |
| U-LoRA adds inference latency relative to LoRA (non-mergeable gating) | Qwen3-8B r=8 · Efficiency trade-off | task latency: 1.00x -> 1.39x | Hurt |
Closest papers by meaning
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation (NeurIPS 2025 · similarity 0.93)
- Uni-LoRA: One Vector is All You Need (NeurIPS 2025 · similarity 0.93)
- SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation (NeurIPS 2026 · similarity 0.92)
- Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape (ICML 2025 · similarity 0.92)
- The Expressive Power of Low-Rank Adaptation (ICLR 2024 · similarity 0.92)
- AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping (NeurIPS 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
