Research map / Large language models / Instruction and fine-tuning

Exploring Model Editing for LLM-based Aspect-Based Sentiment Classification

AAAI 2025 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Uses causal tracing to find mid-layer (10-15) aspect-token representations critical for LLM aspect-based sentiment classification, then fine-tunes only those parts via combined LoRA-style weight editing and representation editing (ReFT-like), matching or beating PEFT baselines with far fewer parameters.

Problem

Adapting large language models to aspect-based sentiment classification (ABSC) with parameter-efficient fine-tuning still updates broad parts of the architecture, causing redundancy; it is unclear which LLM components matter for ABSC sentiment prediction.

Approach

Apply causal tracing (clean, corrupted, and corrupted-with-restoration runs) to locate influential mid-layer aspect-token hidden states, then edit only those: low-rank weight edits on the attention output projection plus a low-rank linear representation intervention at aspect-token positions in layers 10-15.

Key results

• In-domain average accuracy of 87.6% for the proposed method with 0.006% of parameters, versus 86.6% for DoRA and 86.5% for LoRA (0.826% params).
• Out-of-domain average accuracy of 85.4% across 10 cross-domain pairs, versus 84.6% for LoReFT and 85.1% for GPT-4o-mini.
• Editing aspect-token positions in layers 10-15 gave the best results among aspect, last, and random mid-position interventions.

Measured effects reported in the paper

Surprising findings

• Zero-shot Llama2-7b reaches only 61.1% average accuracy, far below fine-tuned models.
• PrefixFT, despite its small parameter count, barely improves over zero-shot Llama2-7b in-domain.

Limitations

• Evaluation limited to four review domains and one base model (Llama-2-7b).
• Out-of-domain results are mixed; the method does not win every pair (e.g., LoRA/LoReFT and GPT-4o-mini are stronger on some).

Experimental setup

Datasets: SemEval 2014 SemEval 2015 SemEval 2016 Device reviews Service reviews

Benchmarks: Device Laptop Restaurant Service (in-domain) 12 out-of-domain cross-domain pairs

Metrics: accuracy

Baselines: GPT-4o-mini Llama2-7b zero-shot BERT DeBERTa Flan-T5 Adapter LoRA DoRA PrefixFT LoReFT

Builds on: ROME LoReFT LoRA causal tracing

Closest papers by meaning

• VERO: Verification and Zero-Shot Feedback Acquisition for Few-Shot Multimodal Aspect-Level Sentiment Classification (AAAI 2025 · similarity 0.86)
• MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA (AAAI 2024 · similarity 0.85)
• Adaptive Layer Sparsity for Large Language Models via Activation Correlation Assessment (NeurIPS 2024 · similarity 0.84)
• You Only Read Once: Constituency-Oriented Relational Graph Convolutional Network for Multi-Aspect Multi-Sentiment Classification (AAAI 2024 · similarity 0.84)
• Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models (NeurIPS 2024 · similarity 0.84)
• Modeling Adaptive Inter-Task Feature Interactions via Sentiment-Aware Contrastive Learning for Joint Aspect-Sentiment Prediction (AAAI 2024 · similarity 0.84)