Research map / Label-efficient and robust learning / Domain adaptation and generalization
Knowledge-Aligned Counterfactual-Enhancement Diffusion Perception for Unsupervised Cross-Domain Visual Emotion Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KCDP, a knowledge-aligned counterfactual-enhanced diffusion framework for unsupervised cross-domain visual emotion recognition (realistic images to stickers), using knowledge-guided LDM features, an MoE predictor, and counterfactual contrastive pseudo-labeling.
Problem
Visual emotion recognition models trained on one domain (e.g., realistic images) generalize poorly to low-resource domains such as stickers, due to emotional expression variability and affective distribution shift under unsupervised domain adaptation.
Approach
Injects domain-agnostic knowledge (captions and subject-verb-object triples) into a latent diffusion U-Net via LoRA-tuned knowledge-guided cross-attention, fuses features with an MoE predictor, and generates target pseudo-labels via a CLIP-based counterfactual contrastive learning scheme that isolates the causal effect of emotion prompts.
Key results
- KCDP achieves 46.16% average accuracy on the DA setting, versus 34.08% for the best domain adaptation baseline UniMoS.
- In the UC setting trained on Emoset, KCDP reaches 83.38 on E, 57.71 on S, and 47.84 on U, outperforming TGCA-PVT by about 11.39 points on Emo8.
- CLIEA with MHA+MLP raises E→S alignment accuracy from 32.07 (vanilla CLIP) to 62.78.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use KCDP instead of the strongest listed baseline | UCDVER domain adaptation · E→S (SER30K) | accuracy: 38.83 -> 62.78 | Helped |
| Use KCDP instead of the strongest listed baseline | UCDVER domain adaptation · E→P (abstract) | accuracy: 31.27 -> 40.55 | Helped |
| Use KCDP instead of the strongest listed baseline | UCDVER domain adaptation · E→A (artphoto) | accuracy: 37.57 -> 51.20 | Helped |
| Use KCDP instead of the strongest listed baseline | UCDVER domain adaptation · S→E (Emoset) | accuracy: 34.68 -> 41.29 | Helped |
| Use KCDP instead of the strongest listed baseline | UCDVER domain adaptation · S→P (abstract) | accuracy: 35.14 -> 38.69 | Helped |
| Use KCDP instead of the strongest listed baseline | UCDVER domain adaptation · S→A (artphoto) | accuracy: 32.80 -> 42.50 | Helped |
| Use KCDP instead of EmoVIT | Universal cross-domain recognition · Emoset | accuracy: 84.02 -> 83.38 | Hurt |
| Use KCDP instead of EmoVIT | Universal cross-domain recognition · EmotionROI, SER30K, abstract, artphoto, and Emo8 | accuracy: 53.87 -> 55.75; 45.66 -> 57.71; 34.68 -> 35.84; 45.14 -> 47.14; 36.89 -> 47.84 | Mixed |
| Add CLIEA with MHA and MLP rather than vanilla CLIP (ablation) | CLIEA · E→S (SER30K) and S→E (Emoset) | accuracy: 32.07 -> 62.78; 26.53 -> 41.29 | Helped |
| Add MLP alongside MHA in CLIEA (ablation) | CLIEA · E→S (SER30K) | accuracy: 57.40 -> 62.78 | Helped |
| Add MHA alongside MLP in CLIEA (ablation) | CLIEA · S→E (Emoset) | accuracy: 39.24 -> 41.29 | Helped |
| Add knowledge triples to captions as diffusion conditions (ablation) | KCDP global-classifier variant · S→E (Emoset) | accuracy: 35.95 -> 36.77 | Helped |
Surprising findings
- Vanilla VLMs such as BLIP2 and LLaVa underperform specialized VER models in cross-domain emotion recognition, and KCDP's gains are not uniform (e.g., EmoVIT is higher on E at 84.02 vs 83.38).
Limitations
- Evaluation relies on a small set of emotion datasets; target domain data is limited (e.g., abstract paintings with 279 images).
- Performance relies on InstructBLIP captions and an SRL parser for triples, so errors in captioning propagate to knowledge features.
Experimental setup
Datasets: EmotionROI Emoset SER30K abstract paintings dataset artphoto Emo8
Benchmarks: Emoset EmotionROI SER30K abstract paintings artphoto Emo8
Metrics: classification accuracy
Baselines: PADCLIP AD-CLIP DAMP UniMoS MDAN LORA-V TGCA-PVT BLIP2 InstructBLIP LLaVa EmoVIT
Builds on: latent diffusion models CLIP LoRA UniMoS DAMP
Closest papers by meaning
- Bridge Then Begin Anew: Generating Target-Relevant Intermediate Model for Source-Free Visual Emotion Adaptation (AAAI 2025 · similarity 0.89)
- Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis (AAAI 2026 · similarity 0.86)
- Customizing Visual Emotion Evaluation for MLLMs: An Open-vocabulary, Multifaceted, and Scalable Approach (ICLR 2026 · similarity 0.86)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.85)
- VidEmo: Affective-Tree Reasoning for Emotion-Centric Video Foundation Models (NeurIPS 2025 · similarity 0.85)
- Catch Your Emotion: Sharpening Emotion Perception in Multimodal Large Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.