Research map / Architectures and efficiency / Pruning and efficient tuning
Diving into Kronecker Adapters: Component Design Matters
Jiayu Bai, Danchen Yu, Zhenyu Liao, TianQi Hou, Feng Zhou, Robert C Qiu, Zenan Ling
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase r1 from 2 to 8 (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 49.93 -> 48.80 | Hurt |
| increase r2 from 2 to 16 (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 49.93 -> 53.93 | Improved |
| increase number of components r from 1 to 4 (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 49.93 -> 54.56 | Improved |
| increase r from 4 to 64 (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 54.56 -> 54.26 | No clear effect |
| raise attainable rank to full (64,64,1) vs rank 4 (ablation) | KronA-like adapter · GSM8k, LLaMA-2-7B | exact match: 49.93 -> 49.00 | No clear effect |
| use r=2 instead of r1=4 at attainable rank 8 (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 49.58 -> 51.58 | Improved |
| increase r1 and r2 together under same budget (2,2,8 to 16,16,8) (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 56.71 -> 56.56 | No clear effect |
| increase r2 from 2 to 16 at r=8 (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 56.71 -> 57.95 | Improved |
| initialize A random-uniform and B zero instead of A zero (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 55.12 -> 56.71 | Improved |
| use Kaiming-uniform instead of Kaiming-normal init (ablation) | CDKA · GSM8k, LLaMA-2-7B | exact match: 52.59 -> 56.71 | Improved |
| replace KronA with CDKA (component design) | KronA · GLUE T5-Base average | accuracy: 84.38 -> 87.30 | Improved |
| replace KronA with CDKA | KronA · GSM8k, LLaMA-2-7B | exact match: 49.00 -> 56.71 | Improved |
Closest papers by meaning
- Fast Inference with Kronecker-Sparse Matrices (ICML 2025 · similarity 0.82)
- Adapters Strike Back (CVPR 2024 · similarity 0.82)
- AdMiT: Adaptive Multi-Source Tuning in Dynamic Environments (CVPR 2025 · similarity 0.82)
- Come Together, But Not Right Now: A Progressive Strategy to Boost Low-Rank Adaptation (ICML 2025 · similarity 0.81)
- LoQT: Low-Rank Adapters for Quantized Pretraining (NeurIPS 2024 · similarity 0.81)
- FALQON: Accelerating LoRA Fine-tuning with Low-Bit Floating-Point Arithmetic (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
