AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

LoFT: Low-Rank Adaptation That Behaves Like Full Fine-Tuning

Nurbek Tastan, Stefanos Laskaridis, Martin Takáč, Karthik Nandakumar, Samuel Horváth

ICLR 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: LoRA variants and theory · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use LoFT (r=16) instead of LoRA (r=16)LLaMA-7B · Commonsense reasoning avgaccuracy: 73.57 -> 76.08Improved
use LoFT (r=16) instead of LoRA (r=16)LLaMA2-7B · Commonsense reasoning avgaccuracy: 68.69 -> 80.46Improved
use LoFT (r=16) instead of DoRA (r=16)LLaMA2-7B · Commonsense reasoning avgaccuracy: 79.71 -> 80.46Improved
use LoFT (r=16) instead of LoRA (r=16)LLaMA3-8B · Commonsense reasoning avgaccuracy: 84.97 -> 85.63Improved
reduce LoFT rank from 16 to 1LLaMA-7B · Commonsense reasoning avgaccuracy: 76.08 -> 72.17Hurt
reduce LoFT rank from 16 to 1LLaMA3-8B · Commonsense reasoning avgaccuracy: 85.63 -> 82.42Hurt
use LoFT (r=16) instead of LoRA (r=16)ViT-Base · 4 image datasets avgaccuracy: 75.46 -> 76.12Improved
use LoFT (r=16) instead of full fine-tuningViT-Base · 4 image datasets avgaccuracy: 75.86 -> 76.12No clear effect
reduce LoFT rank from 16 to 4ViT-Base · 4 image datasets avgaccuracy: 76.12 -> 74.27Hurt
add optimizer state calibration (ablation)LoFT on GPT-2 · WikiText-2perplexityImproved
add alternating updates (ablation)LoFT on GPT-2 · WikiText-2perplexityHurt
add second-moment calibration (ablation)LoFT · commonsense reasoning memory and accuracyaccuracy: ~0.1% gain; memory up to 25.65% vs less than 6% over LoRANo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy