Research map / Vision-language and multimodal / CLIP and zero-shot
Personalized LoRA for Human-Centered Text Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PLoRA, a personalized low-rank adapter that combines LoRA with a user-embedding injection plus a plug-and-play framework (personalized dropout and mutual information maximization) for human-centered sentiment analysis under full, few-shot and zero-shot settings.
Problem
Adapting pre-trained language models to human-centered text understanding is hard because user tokens are million-level and sparse, so per-user LoRA adapters are impractical and cold-start users (few or zero samples) degrade performance.
Approach
Fuse a task-specific LoRA with a personalized knowledge injection (a user embedding passed through a low-rank personal adapter that shares the output projection with the task LoRA), and add personalized dropout and an MIM/distillation term so the model can handle anonymous or new users in zero-shot and few-shot settings in a plug-and-play way.
Key results
- On full-shot IMDB-A, B-PLoRA reaches 57.0% accuracy and FT5-PLoRA 58.7%, matching or beating B-MAA (55.4%) and B-UserAdapter (55.7%) while training fewer parameters (3.1x-3.8x trainable-parameter ratio versus 100x for user adapters).
- In zero-shot settings, FT5-PLoRA (ZS) reaches 70.7% accuracy on YELP-B versus 65.6% for B-UserAdapter.
- Ablation on YELP shows removing both PDropout and MIM drops zero-shot F1 from 65.8 to 59.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine PKI and LoRA in PLoRA (ablation) | BERT · IMDB-A full-shot | F1: 48.9 -> 54.9 | Helped |
| Combine PKI and LoRA in PLoRA (ablation) | BERT · IMDB-A full-shot | F1: 53.2 -> 54.9 | Helped |
| Use PDropout (ablation) | B-PLoRA · IMDB-A full-shot | F1: 54.2 -> 54.9 | Helped |
| Use MIM (ablation) | B-PLoRA · IMDB-A full-shot | F1: 53.9 -> 54.9 | Helped |
| Use both PDropout and MIM (ablation) | B-PLoRA · IMDB-B zero-shot | F1: 41.9 -> 45.1 | Helped |
| Use PDropout (ablation) | B-PLoRA · IMDB-B few-shot | F1: 51.4 -> 53.5 | Helped |
| Use MIM (ablation) | B-PLoRA · IMDB-B few-shot | F1: 51.7 -> 53.5 | Helped |
| Use few-shot adaptation rather than direct B-only optimization | B-PLoRA · IMDB-B few-shot | F1: 47.3 -> 53.5 | Helped |
| Use PLoRA (ZS) instead of B-UserAdapter | BERT · GDRD-B zero-shot | F1: 51.5 -> 53.7 | Helped |
| Use PLoRA (ZS) instead of B-UserAdapter | BERT · PPR-B zero-shot | F1: 43.6 -> 42.8 | Hurt |
| Use PLoRA (FS) instead of B-UserAdapter (FS) | BERT · PPR-B few-shot | F1: 44.8 -> 43.7 | Hurt |
Surprising findings
- Without PDropout and MIM, zero-shot performance of PLoRA degrades catastrophically (IMDB-B F1 41.9 vs 45.1), even below plain LoRA trained on generic data.
- Full fine-tuning of B-PKI scores lower than the LoRA-based variants, suggesting FFT is needed for PKI-only adaptation to be effective.
Limitations
- Evaluated only on personalized sentiment analysis with four datasets and small base PLMs (base-size models).
- Long reviews are truncated to first and last 256 tokens, which may lose information.
- Zero-shot results still lag full-shot and few-shot performance, and the PDropout/MIM hyperparameters must be tuned per dataset.
Experimental setup
Datasets: IMDB YELP GDRD PPR
Benchmarks: IMDB-A/B YELP-A/B GDRD-A/B PPR-A/B
Metrics: Accuracy MSE macro F1
Baselines: NSC NSC+U MA-BERT (MAA) UserAdapter BERT full fine-tuning B-LoRA B-PKI
Builds on: LoRA UserAdapter MA-BERT
Closest papers by meaning
- Text-to-LoRA: Instant Transformer Adaption (ICML 2025 · similarity 0.86)
- Uni-LoRA: One Vector is All You Need (NeurIPS 2025 · similarity 0.85)
- Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment (ICML 2025 · similarity 0.84)
- LoRA-One: One-Step Full Gradient Could Suffice for Fine-Tuning Large Language Models, Provably and Efficiently (ICML 2025 · similarity 0.84)
- Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering (ICLR 2025 · similarity 0.84)
- KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.