AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

Understanding LoRA as Knowledge Memory: An Empirical Analysis

Seungju Back, Dongwoo Lee, Naun Kang, Taehee Lee, S. K. Hong, Youngjune Gwon, Sungjin Ahn

ICML 2026 · Large language models › Retrieval and knowledge · approach: Retrieval-augmented reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train LoRA on synthetic QA instead of raw text (ablation)LoRA memory, Llama-3.1-8B · PaperQALLM-judge score: 3.187 -> 5.893Improved
mix Original, Summary8, Rewrite4 and QA40 instead of QA40 alone (ablation)LoRA memory, Llama-3.1-8B · PaperQALLM-judge score: 5.893 -> 6.822Improved
mix Summary8 with QA40 instead of QA40 alone (ablation)LoRA memory, Qwen3-8B · PaperQALLM-judge score: 5.409 -> 5.662Improved
scale base model from 1.7B to 8B (ablation)LoRA memory, Qwen3 · PaperQALLM-judge scoreNo clear effect
generate synthetic data with GPT-4.1 instead of Llama-3.1-8B (ablation)LoRA memory · PaperQALLM-judge scoreImproved
increase LoRA rank (ablation)single LoRA · CounterFactefficacyImproved
use highest rank instead of low rank (ablation)single LoRA · PhoneBook and CounterFactparameter efficiencyHurt
partition knowledge across multiple small LoRAs with oracle routing (ablation)multi-LoRA · PhoneBook 64Kexact matchImproved
replace oracle routing with embedding-based routing (ablation)multi-LoRA · PaperQALLM-judge scoreHurt
merge LoRAs with TIES instead of DARE or linear (ablation)multi-LoRA merging · PaperQALLM-judge scoreImproved
rescale concatenation by 1/sqrt(N) (ablation)multi-LoRA merging (CAT) · PaperQALLM-judge scoreImproved
merge more LoRAs (N from 1 to 5) with TIES under ground-truth selection (ablation)multi-LoRA · PaperQALLM-judge scoreHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy