AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

SaMA: Morpho Adaptation via Asymmetric Expansion of Kronecker Product

An Nguyen, Anh Tong

NeurIPS 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: LoRA variants and theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SaMA, a PEFT method that uses an asymmetric expansion of Kronecker products to overcome LoRA's flat-direction optimization bottleneck and linear rank scaling with parameter budget.

Problem

LoRA's invariance manifold creates flat directions in the loss landscape that hinder adaptive optimizers, and its effective rank grows only linearly with the parameter budget.

Approach

Replaces LoRA's low-rank update with an asymmetric expansion based on Kronecker products, which removes the invariance manifold responsible for flat directions and lets effective rank grow faster than linearly in the parameter budget.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase LoRA rank from 32 to 128 (ablation)LoRA · DROP (Llama2-7B)F1: 62.40 -> 58.82Hurt
use SaMA instead of LoRA at similar parameter budget (0.062%)Llama2-7B · DROPF1: 62.39 -> 66.61Improved
scale SaMA block size from 2,2 to 64,64 (ablation)SaMA · DROP (Llama2-7B)F1: 66.01 -> 68.07Improved
SaMA instead of QuanTA on Llama2-13BLlama2-13B · DROPF1: 69.0 -> 73.12Improved
use SaMA instead of ABBAMistral-7B · GSM8Kaccuracy: 64.48 -> 67.70Improved
use SaMA 8,8 instead of ABBA on MATHGemma-2 9B · MATHaccuracy: 39.59 -> 39.36Hurt
use SaMA 4x4 instead of QuanTA with Q-V tuningLLaMA-7B · Commonsense reasoningaverage accuracy: 80.3 -> 81.9Improved
increase SaMA from 4x4 to 16x16 (ablation)SaMA · Commonsense reasoning (LLaMA3-8B)average accuracy: 87.4 -> 87.8Improved
increase SaMA from 4x4 to 16x16 (ablation)SaMA · Commonsense reasoning (LLaMA-13B)average accuracy: 85.5 -> 85.4No clear effect
expand local gauge group to dense 16-by-16 blocks (ablation)SaMA · GSM8K and MATHaccuracyHurt
initialize L=0 and R random instead of R=0 and L random (ablation)SaMA · DROP (Qwen2.5-1.5B)F1: 39.41 -> 39.77Improved
use SaMA 2-2 instead of strongest PEFT baselinesViT-B/16 · VTAB-1kmean total accuracy: 74.5 -> 74.7Improved

Experimental setup

Baselines: LoRA

Builds on: LoRA

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy