Research map / Label-efficient and robust learning / Self-supervised and contrastive
FEEL: Quantifying Heterogeneity in Physiological Signals for Generalizable Emotion Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Benchmarks 16 EDA/PPG emotion-recognition architectures across 19 public datasets (within- and cross-dataset), finding fine-tuned CLSP models and handcrafted-feature models strongest, with generalization depending on setting, device, and labeling.
Problem
Physiological (EDA/PPG) emotion recognition lacks standardized, large-scale evaluation across heterogeneous datasets, limiting progress and cross-dataset generalization.
Approach
Unifies 19 heterogeneous public EDA/PPG emotion datasets via a common preprocessing and binary/four-class arousal-valence binning, benchmarks four modeling paradigms, and adapts CLSP pretrained signal-language models with a CoCoOp-style meta-net for fine-tuning without text labels, then analyzes cross-dataset transfer along setting, device, and labeling dimensions.
Key results
- Fine-tuned CLSP models were best in 71 of 114 top-performing binary instances; handcrafted-feature models won 107 of 114.
- Models trained on real-life data transferred to lab (F1 = 0.79) and constraint settings (F1 = 0.78).
- Models trained on lab-based devices transferred to custom wearable (F1 = 0.81) and Empatica E4 (F1 = 0.73).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use fine-tuned CLSP models rather than other evaluated model paradigms | Binary emotion classification benchmark · 19-dataset benchmark, arousal and valence | top-performing model instances: 71 of the 114 | Helped |
| Use handcrafted features rather than raw signal segments | Evaluated emotion-recognition models · 19-dataset benchmark | top-performing model instances: 107/114 | Helped |
| Use EDA rather than other modalities | Evaluated emotion-recognition models · Arousal classification across datasets | datasets with top performance: 12 out of 19 | Helped |
| Use EDA rather than other modalities | Evaluated emotion-recognition models · Valence classification across datasets | datasets with top performance: 13 out of 19 | Helped |
| Fine-tune CLSP on target data rather than use zero-shot CLSP | CLSP models · Binary classification across datasets | top-performing cases: 14 cases | Helped |
| Use CLSP+CNN rather than other CLSP variants | CLSP models · Binary classification across datasets | top-performing cases: 37 top-performing cases | Helped |
| Train on real-world data and transfer to lab or constraint settings | Cross-setting models · Lab and constraint settings | F1: 0.79; 0.78 | Helped |
| Train on constraint-based data and transfer to real-life data | RF · Real-life setting | F1: 0.88 | Helped |
| Train on lab-based devices and transfer to custom wearable or E4 | Cross-device models · Custom wearable and Empatica E4 | F1: 0.81; 0.73 | Helped |
| Train on expert-annotated data and transfer to stimulus-labeled or self-reported data | Cross-labeling models · Stimulus-labeled and self-reported datasets | F1: 0.72; 0.76 | Helped |
| Train on stimulus-labeled data and transfer to self-reported data | Cross-labeling models · Self-reported datasets | F1: 0.63 | Hurt |
| Use EDA rather than PPG or EDA+PPG in zero-shot CLSP transfer | CLSP zero-shot models · Arousal, cross-gender transfer | F1 | Helped |
Surprising findings
- Simple models such as Random Forests, LDA and MLP remain competitive, winning 36 of 114 top instances.
- Zero-shot CLSP models using EDA achieved the highest overall F1 (0.91) on expert-labeled data.
Limitations
- Advanced or physiology-specific architectures were not benchmarked.
- Harmonization lacks a common labeling technique; binarized labels are heuristic.
- Cultural context and health status heterogeneity not analyzed.
Experimental setup
Datasets: WESAD NURSE EMOGNITION UBFC_PHYS VERBIO PhyMER EmoWear MAUS CLAS CASE Unobtrusive CEAP-360VR
Benchmarks: WESAD NURSE EMOGNITION UBFC_PHYS VERBIO PhyMER EmoWear MAUS CLAS CASE Unobtrusive CEAP-360VR
Metrics: F1 accuracy
Baselines: Random Forest LDA MLP ResNet LSTM Attention-based model CLSP zero-shot
Builds on: CLSP CoCoOp SMOTE
Closest papers by meaning
- EEVR: A Dataset of Paired Physiological Signals and Textual Descriptions for Joint Emotion Representation Learning (NeurIPS 2024 · similarity 0.89)
- AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models (ICML 2025 · similarity 0.88)
- egoEMOTION: Egocentric Vision and Physiological Signals for Emotion and Personality Recognition in Real-world Tasks (NeurIPS 2025 · similarity 0.87)
- Multi-dataset Joint Pre-training of Emotional EEG Enables Generalizable Affective Computing (NeurIPS 2025 · similarity 0.87)
- OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition (ICML 2025 · similarity 0.87)
- Beyond Sensor Data: Foundation Models of Behavioral Data from Wearables Improve Health Predictions (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.