Research map / Vision-language and multimodal / Human and embodied modeling
Multiple Feature Refining Network for Visual Emotion Distribution Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MFRN, combining wavelet-based frequency modeling (Spectral Mixer) for low-level features and graph prompt learning over object/attribute graphs for high-level emotional semantics, for visual emotion distribution learning.
Problem
Visual emotion distribution learning (VEDL) suffers from redundant, semantic-irrelevant information in features from unbiased filtering, limiting capture of salient and subtle emotions in images.
Approach
Refines features by filtering information from two levels: DTCWT-based separated low/high-frequency modeling for low-level emotional saliency, and a semantic graph over detected objects and attributes with learnable unified graph prompts for emotion-oriented high-level semantics, trained with KL distribution loss plus a dominant-emotion cross-entropy loss.
Key results
- MFRN achieves the best average rank on Emotion6, Flickr-LDL and Twitter-LDL (rank 1.00, 1.17, 1.83 respectively) in Table 1.
- On Flickr-LDL, MFRN reaches KL 0.3898 and cosine 0.9168 versus StyleEDL's 0.36/0.84 in the reported table (KL 0.39 for the best baseline in the F block).
- Unified graph prompt outperforms object, attribute and external prompt variants on Flickr-LDL (KL 0.38, Cheb 0.2001, Cosine 0.9142 for UP vs. OP KL 0.3543 and AP KL 0.4771).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add channel-wise learning to spatial features (ablation) | Spectral Mixer · Flickr-LDL | KL divergence: 0.4352 -> 0.3808 | Helped |
| combine Spectral Mixer and Graph Prompt Learning rather than use either alone (ablation) | MFRN · Twitter-LDL | KL divergence: 0.6483 -> 0.4517; 0.6266 -> 0.4517 | Helped |
| add Semantic Calibration loss (ablation) | MFRN · Flickr-LDL | KL divergence: 0.4528 -> 0.3898 | Helped |
| prompt all graph nodes rather than use Object Prompt (ablation) | Semantic Graph Prompt Learning · Flickr-LDL | KL divergence: 0.3543 -> 0.38 | No clear effect |
| prompt all graph nodes rather than use Attribute Prompt (ablation) | Semantic Graph Prompt Learning · Flickr-LDL | KL divergence: 0.4771 -> 0.38 | Hurt |
| prompt all graph nodes rather than use External Prompt (ablation) | Semantic Graph Prompt Learning · Flickr-LDL | KL divergence: 0.3247 -> 0.38 | Hurt |
| use MFRN instead of StyleEDL | VEDL · Emotion6 | Avg Rank: 2.17(2) -> 1.00(1) | Helped |
| use MFRN instead of StyleEDL | VEDL · Flickr-LDL | Avg Rank: 2.17(2) -> 1.17(1) | Helped |
| use MFRN instead of StyleEDL | VEDL · Twitter-LDL | KL divergence: 0.42 -> 0.45 | Hurt |
| use MFRN instead of StyleEDL | VEDL · Twitter-LDL | Chebyshev distance: 0.22 -> 0.25 | Hurt |
Surprising findings
- Attribute-only prompting performs poorly compared with object prompting, and the graph prompt alone (GPL-only) is far weaker than the combined model.
- Improvement on Twitter-LDL is less pronounced than on other datasets, which the authors attribute to abstract, less explicit image semantics.
Limitations
- Performance gain on Twitter-LDL is modest due to abstract image semantics.
- Parameter count (122M) is larger than several variants and relies on an external object detector.
- No consensus on prompt injection methodology; design choices were found empirically.
Experimental setup
Datasets: Emotion6 Flickr-LDL Twitter-LDL
Benchmarks: Emotion6 Flickr-LDL Twitter-LDL
Metrics: Chebyshev distance Clark distance Canberra metric KL divergence cosine similarity intersection similarity average rank
Baselines: PT-Bayes PT-SVM AA-kNN AA-BP SA-BFGS SA-CPNN SSDL LDL-LDM DIEDL JointLDL StyleEDL EAP
Builds on: Dual-Tree Complex Wavelet Transform Vision Transformer Faster R-CNN bottom-up attention StyleEDL
Closest papers by meaning
- MDAN: Multi-Level Dependent Attention Network for Visual Emotion Analysis (CVPR 2022 · similarity 0.88)
- A Circular-Structured Representation for Visual Emotion Distribution Learning (CVPR 2021 · similarity 0.87)
- Catch Your Emotion: Sharpening Emotion Perception in Multimodal Large Language Models (ICML 2025 · similarity 0.85)
- Bridge Then Begin Anew: Generating Target-Relevant Intermediate Model for Source-Free Visual Emotion Adaptation (AAAI 2025 · similarity 0.85)
- Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition (CVPR 2025 · similarity 0.84)
- Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.