AI Paper Atlas

Research map / Architectures and efficiency / Quantization

Efficient LLM Adaptation with Forward-Only Passes

Baichuan Huang, Ananth Balashankar, Amir Aminifar

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a forward-only approach that adapts LLMs to tasks using only inference-time forward passes, exploiting angle concentration of activations, instead of backpropagation.

Problem

Efficient task-specific adaptation of LLMs is costly because standard fine-tuning requires backpropagation; the paper asks whether the forward passes used in inference serving can enable adaptation.

Approach

Uses forward-only passes already executed during LLM inference serving to perform task-specific adaptation, leveraging the angle concentration of activations induced by each (input) to avoid backward computation.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use forward-only search instead of gradient-based LASER-Grad to pick matrixrank reduction · GPT-J, 8 datasetspeak memory (GB): 22.0 -> 11.5Improved
Use hierarchical forward-only search instead of exhaustive LASER (std)rank reduction · GPT-J, 8 datasetsspeedup: 1 -> 36.7xImproved
Use forward-only search instead of LASER (std)rank reduction · GPT-J, 8 datasetsaverage accuracy gain: 8.24 -> 11.3Improved
Use forward-only search instead of LASER (std)rank reduction · GPT-J Epistemic Reasoningaccuracy (%): 38.3 -> 62.8Improved
Use forward-only search instead of LASER (std)rank reduction · GPT-J Bios Professionaccuracy (%): 82.1 -> 79.1Hurt
Use hierarchical search instead of std evaluation over full validation (ablation)forward-only search · GPT-J, 8 datasetsspeedup: 5.6x -> 36.7xImproved
Use 100-sample subset instead of std validation set (ablation)forward-only search · GPT-J, 8 datasetsaverage accuracy gain: 11.2 -> 9.7Hurt
Use top-j=5 instead of j=10 (ablation)hierarchical search · GPT-JaccuracyHurt
Use last m=20 instead of m=50 SVD components (ablation)hierarchical search · GPT-JaccuracyMixed
Increase last-m SVD components (20, 50, 100) (ablation)forward-only search · GPT-Jpeak memoryHurt
Reduce D_subset size from 100 to 10, 5, 1 (ablation)hierarchical search · RoBERTa-basefinal candidateNo clear effect
Add attention matrices (key, query, value, out) to candidates (ablation)forward-only search · RoBERTa-basetop-j matricesNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy