AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

DiaBlo: Diagonal Blocks Are Sufficient For Finetuning

Selcuk Gurses, Aozhong Zhang, Yanxia Deng, Xun Dong, Xin Li, Naigang Wang, Penghang Yin, Zi Yang

ICLR 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Adapter-based parameter-efficient tuning · 4 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Fine-tune diagonal blocks (DiaBlo) instead of LoRALoRA r=32 · commonsense reasoning, Llama2-7Bavg accuracy: 77.6 -> 83.5Improved
Use DiaBlo instead of SMTSMT · commonsense reasoning, Llama2-7Bavg accuracy: 81.8 -> 83.5Improved
Use DiaBlo instead of full fine-tuningfull FT · commonsense reasoning, Llama3-8Bavg accuracy: 87.5 vs 87.3No clear effect
Use DiaBlo instead of LoRA on arithmetic reasoningLoRA r=64 · GSM8K/MATH avg, Llama2-7Bavg exact match: 38.7 -> 43.4Improved
Use N=64 instead of N=32 diagonal blocks (ablation)DiaBlo · MATH, Llama2-7Bexact match: 20.4 -> 17.6Hurt
Use DiaBlo instead of LoRILoRI · HumanEval Pass@1, Llama3-8BPass@1: 43.2 vs 43.2No clear effect
Use DiaBlo instead of DoRA/LoRA for safety alignmentLoRA · HEx-PHI, Llama3-8Brefusal rate: 91.6 -> 97.6Improved
Replace LoRA with DiaBlo on MagR-quantized 2-bit modelMagR-LoRA · arithmetic avg, Llama2-7B 2-bitavg accuracy: 46.2 -> 48.7Improved
Replace LoRA with DiaBlo on 4-bit Llama2-7BQLoRA · arithmetic avg, 4-bitavg accuracy: 53.7 -> 54.8Improved
Use diagonal blocks instead of random entries as sparsity pattern (ablation)DiaBlo · GSM8K fine-tuning, Llama3-8Baccuracy: 65.35 -> 67.68Improved
Use diagonal blocks instead of random rows (ablation)DiaBlo · GSM8K retained update at 1/128accuracy: 2.85 -> 63.11Improved
Use DiaBlo instead of DoRADoRA · training time per epochminutes: 480 -> 170Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy