Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
EEVR: A Dataset of Paired Physiological Signals and Textual Descriptions for Joint Emotion Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EEVR, a dataset pairing PPG/EDA signals from 37 participants watching 360° VR videos with self-reported textual descriptions, and CLSP, a contrastive language-signal pre-training method evaluated with LOSO and zero-shot transfer.
Problem
Physiological emotion datasets rely on objective scales or stimulus-based labels that miss nuanced, mixed, or absent emotions; no prior dataset pairs physiological signals with raw self-reported text for emotion recognition.
Approach
Collects semi-structured qualitative interview text describing each participant's emotions per VR stimulus, aligned with PPG and EDA segments, and uses contrastive language-signal pre-training (CLSP) to align signal features with text embeddings from DistilBERT for valence/arousal classification.
Key results
- Text-only DistilBERT reaches 91.73% arousal and 97.44% valence accuracy, while EDA-only handcrafted features with random forest reach about 90.8% valence accuracy.
- CLSP with EDA+Text improves arousal accuracy to 70.38% and valence to 64.19% in the EEVR experiments, versus about 61.86% and 87.39% for EDA-only HC+NN baseline (arousal/valence columns as reported in Table 3).
- Zero-shot CLSP reaches 84.59% valence accuracy on NURSE (EDA+PPG) versus 76.04% for supervised MLP, and 75.00% arousal accuracy on WESAD (EDA+PPG) versus 91.67% for MLP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add paired textual descriptions using CLSP to EDA features | HC+NN physiological baseline · EEVR | accuracy (valence): 61.86 -> 70.38 | Helped |
| Add paired textual descriptions using CLSP to EDA features | HC+NN physiological baseline · EEVR | accuracy (arousal): 57.27 -> 77.25 | Helped |
| Add paired textual descriptions using CLSP to PPG features | HC+NN physiological baseline · EEVR | accuracy (valence): 56.76 -> 64.74 | Helped |
| Add paired textual descriptions using CLSP to PPG features | HC+NN physiological baseline · EEVR | accuracy (arousal): 54.16 -> 69.91 | Helped |
| Add paired textual descriptions using CLSP to EDA+PPG features | HC+NN physiological baseline · EEVR | accuracy (valence): 58.86 -> 64.87 | Helped |
| Add paired textual descriptions using CLSP to EDA+PPG features | HC+NN physiological baseline · EEVR | accuracy (arousal): 58.58 -> 69.64 | Helped |
| Add paired textual descriptions using CLSP to EDA features | HC+NN physiological baseline · EEVR | accuracy (stimulus-label): 87.39 -> 64.19 | Hurt |
| Use zero-shot CLSP instead of supervised MLP | Emotion recognition model · Emognition (EDA) | accuracy (arousal): 52.80 -> 53.23 | Helped |
| Use zero-shot CLSP instead of supervised MLP | Emotion recognition model · Emognition (PPG) | accuracy (valence): 50.63 -> 51.88 | Helped |
| Use zero-shot CLSP instead of supervised MLP | Emotion recognition model · Nurse (EDA) | accuracy (arousal): 39.88 -> 55.48 | Helped |
| Use zero-shot CLSP instead of supervised MLP | Emotion recognition model · Nurse (PPG) | accuracy (arousal): 45.10 -> 53.08 | Helped |
| Use zero-shot CLSP instead of supervised MLP | Emotion recognition model · Nurse (EDA + PPG) | accuracy (arousal): 48.35 -> 53.08 | Helped |
Surprising findings
- Text-only classifiers outperform physiological-signal baselines by a wide margin, raising concern that text may leak labels.
- Zero-shot CLSP underperforms the supervised MLP on WESAD in several settings despite strong results on NURSE.
Limitations
- Small sample of 37 participants from a privileged, educated demographic limits generalization.
- Controlled lab setting and VR sickness/familiarity may not reflect real-life emotional responses.
- Arousal labels are skewed and required oversampling; labels derive from participant self-report, which is susceptible to bias.
Experimental setup
Datasets: EEVR Emognition WESAD NURSE
Benchmarks: EEVR Emognition WESAD NURSE
Metrics: accuracy F1 score
Baselines: Logistic Regression Decision Tree Random Forest LDA XGBoost SVM MLP XLM-RoBERTa Base HC+NN
Builds on: CLIP DistilBERT cvxEDA NeuroKit
Closest papers by meaning
- FEEL: Quantifying Heterogeneity in Physiological Signals for Generalizable Emotion Recognition (NeurIPS 2025 · similarity 0.89)
- Multi-dataset Joint Pre-training of Emotional EEG Enables Generalizable Affective Computing (NeurIPS 2025 · similarity 0.87)
- $E^3$: Exploring Embodied Emotion Through A Large-Scale Egocentric Video Dataset (NeurIPS 2024 · similarity 0.86)
- Nano-EmoX: Unifying Multimodal Emotional Intelligence from Perception to Empathy (CVPR 2026 · similarity 0.85)
- EmoVIT: Revolutionizing Emotion Insights with Visual Instruction Tuning (CVPR 2024 · similarity 0.84)
- DMMR: Cross-Subject Domain Generalization for EEG-Based Emotion Recognition via Denoising Mixed Mutual Reconstruction (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.