Research map / Vision-language and multimodal / Human and embodied modeling
Aspect Enhancement and Text Simplification in Multimodal Aspect-Based Sentiment Analysis for Multi-Aspect and Multi-Sentiment Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AETS, a multimodal aspect-based sentiment model that adds a masking-based aspect enhancement module and a syntax-driven (BFS over dependency graph) text simplification module, evaluated on MATE, MASC and JMASA on Twitter-2015/2017.
Problem
Multimodal aspect-based sentiment analysis struggles with multi-aspect and multi-sentiment sentences: aspect terms are missed in multimodal data and aspects are poorly associated with their sentiment words.
Approach
Mask-based aspect enhancement that highlights noun candidate aspects in text and image representations, plus a dependency-graph BFS text simplification that extracts each aspect's nearby adjective/sentiment phrase and appends it to the input for sentiment classification.
Key results
- On JMASA, AETS reaches 72.1 F1 on Twitter-2015 and 73.1 F1 on Twitter-2017, versus 68.6 and 69.7 for the prior best AoM.
- On MATE, AETS reaches 90.9 F1 on Twitter-2015 and 95.3 F1 on Twitter-2017, above AoM and CMMT.
- On MASC, AETS gets 80.8 F1 on Twitter-2015 and 75.2 F1 on Twitter-2017, but its Twitter-2015 accuracy (79.5) is below AoM (80.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AETS instead of AoM | JMASA · Twitter-2015 | F1: 68.6 -> 72.1 | Helped |
| Use AETS instead of AoM | JMASA · Twitter-2017 | F1: 69.7 -> 73.1 | Helped |
| Use AETS instead of AoM on Twitter-2015 and CMMT on Twitter-2017 | MATE · Twitter-2015 and Twitter-2017 | F1: 86.2 -> 90.9; 93.1 -> 95.3 | Helped |
| Use AETS instead of AoM | MASC · Twitter-2015 and Twitter-2017 | F1: 75.9 -> 80.8; 75.0 -> 75.2 | Helped |
| Add image information (ablation) | AETS · Twitter-2015 and Twitter-2017 | F1: 70.4 -> 72.1; 70.7 -> 73.1 | Helped |
| Add multi-head attention (ablation) | AETS · Twitter-2015 and Twitter-2017 | F1: 68.4 -> 72.1; 70.2 -> 73.1 | Helped |
| Add text-based candidate aspect attention (ablation) | AETS · Twitter-2015 and Twitter-2017 | F1: 64.9 -> 72.1; 64.5 -> 73.1 | Helped |
| Add image-based candidate aspects (ablation) | AETS · Twitter-2015 and Twitter-2017 | F1: 71.2 -> 72.1; 68.8 -> 73.1 | Helped |
| Add text simplification (ablation) | AETS · Twitter-2015 and Twitter-2017 | F1: 70.2 -> 72.1; 66.7 -> 73.1 | Helped |
Surprising findings
- On MASC Twitter-2015, AETS has lower accuracy than AoM (79.5 vs 80.2) even though it has higher F1.
- Removing text simplification drops Twitter-2017 precision by 8.9%, the largest ablation effect reported.
Limitations
- Gains on Twitter-2015 MASC accuracy are not achieved; the authors attribute this to few multi-aspect multi-sentiment samples.
- Text simplification relies on spaCy dependency parsing and adjective-based stopping heuristics, which may miss sentiment not encoded by adjectives.
- Evaluated only on two Twitter datasets.
Experimental setup
Datasets: Twitter-2015 Twitter-2017
Benchmarks: Twitter-2015 Twitter-2017
Metrics: Precision Recall Micro-F1 Accuracy
Baselines: SPAN D-GCN RoBERTa BART UMT+TomBERT OSCGA+TomBERT JML VLP-MABSA CMMT Atlantis AoM RAN
Builds on: RoBERTa ViT spaCy VLP-MABSA AoM
Closest papers by meaning
- DEQA: Descriptions Enhanced Question-Answering Framework for Multimodal Aspect-Based Sentiment Analysis (AAAI 2025 · similarity 0.90)
- A Novel Energy Based Model Mechanism for Multi-Modal Aspect-Based Sentiment Analysis (AAAI 2024 · similarity 0.88)
- Aspect-Based Sentiment Analysis with Explicit Sentiment Augmentations (AAAI 2024 · similarity 0.85)
- You Only Read Once: Constituency-Oriented Relational Graph Convolutional Network for Multi-Aspect Multi-Sentiment Classification (AAAI 2024 · similarity 0.84)
- SimRP: Syntactic and Semantic Similarity Retrieval Prompting Enhances Aspect Sentiment Quad Prediction (AAAI 2025 · similarity 0.84)
- TextGT: A Double-View Graph Transformer on Text for Aspect-Based Sentiment Analysis (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.