Research map / Large language models / Instruction and fine-tuning
ReFT: Representation Finetuning for Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Representation Finetuning (ReFT), which trains low-rank interventions on frozen LM hidden representations as a PEFT alternative, instantiated as LoReFT and DiReFT.
Problem
Parameter-efficient finetuning methods adapt LLMs by updating weights, whereas interpretability work suggests representations carry rich semantics that could be edited more efficiently and powerfully.
Approach
Define ReFT as learned interventions on hidden representations at chosen layers and positions of a frozen LM; LoReFT edits the representation within a learned low-rank linear subspace (building on distributed alignment search) and DiReFT is a cheaper LoRA-like ablation applied directly to representations.
Key results
- LoReFT on LLaMA-1 7B reaches 80.2% average accuracy on commonsense reasoning vs. 78.1% for DoRA, using 0.031% of parameters.
- LoReFT on Llama-2 7B achieves 85.60% win-rate on AlpacaEval v1.0, above full finetuning (80.93%) and LoRA (81.48%) with 0.0039% parameters.
- On arithmetic reasoning, LoReFT (42.6 avg on LLaMA-7B) trails LoRA (46.9), though it outperforms prefix-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LoReFT instead of the strongest listed PEFT baseline | LLaMA-family models · Commonsense reasoning (8-task Avg.) | accuracy: 78.1 -> 80.2; 81.5 -> 83.3; 80.5 -> 81.8; 85.2 -> 86.6 | Helped |
| Remove LoReFT orthogonality constraint and difference operation (DiReFT) (ablation) | LoReFT · Commonsense reasoning (8-task Avg.) | accuracy: 80.2 -> 79.0; 83.3 -> 82.2; 81.8 -> 80.9; 86.6 -> 85.4 | Hurt |
| Use LoReFT instead of LoRA | LLaMA-family models · Arithmetic reasoning (4-task Avg.) | accuracy: 46.9 -> 42.6; 51.1 -> 49.6 | Hurt |
| Remove LoReFT orthogonality constraint and difference operation (DiReFT) (ablation) | LoReFT · Arithmetic reasoning (4-task Avg.) | accuracy: 42.6 -> 40.6; 49.6 -> 48.0 | Hurt |
| Use LoReFT instead of LoRA | Llama-2 7B · Alpaca-Eval v1.0 | win-rate: 81.48 -> 85.60 | Helped |
| Remove LoReFT orthogonality constraint and difference operation (DiReFT) (ablation) | LoReFT · Alpaca-Eval v1.0 | win-rate: 85.60 -> 84.85 | Hurt |
| Halve LoReFT rank (ablation) | LoReFT on Llama-2 7B · Alpaca-Eval v1.0 | win-rate: 85.60 -> 84.12 | Hurt |
| Train LoReFT with 1K examples (ablation) | LoReFT on Llama-2 7B · Alpaca-Eval v1.0 | win-rate: 85.60 -> 81.91 | Hurt |
| Use LoReFT instead of RED | RoBERTa-base · GLUE (Avg.) | accuracy: 84.3 -> 84.2 | Hurt |
| Use LoReFT instead of LoRA | RoBERTa-large · GLUE (Avg.) | accuracy: 88.1 -> 88.2 | Helped |
| Remove LoReFT orthogonality constraint and difference operation (DiReFT) (ablation) | LoReFT · GLUE (Avg.) | accuracy: 84.2 -> 83.2; 88.2 -> 87.4 | Hurt |
Surprising findings
- On arithmetic reasoning, ReFT methods underperform LoRA and adapters, which the authors attribute to long generations diluting intervention effects.
- A rank-1 LoReFT on RoBERTa-base with a single intervention layer matches PEFT baselines on GLUE despite very few parameters.
Limitations
- Explored mainly the LLaMA family; other model families and vision-language models left to future work.
- Large hyperparameter search space makes the method's full capabilities unclear.
- Weaker on chain-of-thought arithmetic reasoning than LoRA.
Experimental setup
Datasets: Commonsense170K Math10K UltraFeedback Alpaca-52K GLUE GSM8K
Benchmarks: BoolQ PIQA SIQA HellaSwag WinoGrande ARC-e ARC-c OBQA AQuA GSM8K MAWPS SVAMP
Metrics: accuracy win-rate parameter percentage
Baselines: LoRA DoRA Prefix-tuning Series Adapter Parallel Adapter BitFit RED VeRA full finetuning ChatGPT GPT-3.5 Turbo 1106
Builds on: distributed alignment search distributed interchange intervention LoRA DoRA pyvene
Closest papers by meaning
- Bias-Restrained Prefix Representation Finetuning for Mathematical Reasoning (AAAI 2026 · similarity 0.90)
- RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models (NeurIPS 2025 · similarity 0.88)
- Efficient Post-Training Refinement of Latent Reasoning in Large Language Models (AAAI 2026 · similarity 0.88)
- UFT: Unifying Supervised and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.87)
- LoFiT: Localized Fine-tuning on LLM Representations (NeurIPS 2024 · similarity 0.87)
- Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.