Research map / Federated learning and privacy / Privacy attacks and secure learning
Privacy Enhanced Multimodal Neural Representations for Emotion Recognition
Mimansa Jaiswal, Emily Mower Provost
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adversarially train representations to hide gender | Emotion recognition models · Multiple datasets and modalities | privacy metric: privacy metric is always higher when the representations are trained adversarially | Improved |
| adversarially train representations to hide gender | Emotion recognition models · Multiple datasets and modalities | emotion recognition UAR | No clear effect |
| adversarially train representations to hide gender | Unimodal lexical emotion models · MuSE and IEMOCAP | emotion recognition UAR | Improved |
| increase adversarial strength from λ=0.50 to λ=0.75 (ablation) | Unimodal acoustic MuSE · MuSE | privacy metric: 0.39 to 0.45 | Improved |
| increase adversarial strength from λ=0.75 to λ=1.00 (ablation) | Unimodal acoustic MuSE · MuSE | privacy metric: 0.45 to 0.41 | Hurt |
| apply separate adversarial components after pooling each modality (ablation) | Multimodal emotion recognition · MuSE and IEMOCAP | privacy metric: Iem - U: 0.66, P: 0.43; MuS - U: 0.63, P: 0.44; Valence: Iem - U: 0.67, P: 0.46; MuS - U: 0.64, P: 0.46 | Improved |
| replace gender invariance with speaker-identity invariance (ablation) | Emotion recognition models · Multiple datasets | membership identification UAR: Activation: Imp - 0.58, Pod - 0.60, Iem - 0.58, MuS - 0.62; Valence: Imp - 0.54, Pod - 0.56, Iem - 0.59, MuS - 0.60 | Improved |
| add speaker-identity invariance alongside gender invariance (ablation) | Emotion recognition models · Multiple datasets and modalities | membership identification UAR: 0.58 | Improved |
| add speaker-identity invariance alongside gender invariance (ablation) | Emotion recognition models · MuSE activation | emotion recognition UAR: 0.66 to 0.65 | Hurt |
| use multimodal rather than unimodal representations | General emotion recognition models · MuSE and IEMOCAP | gender leakage: Activation: Iem 0.74, MuS 0.73; valence: Iem 0.68, MuS 0.64 | Hurt |
Closest papers by meaning
- MultiPriv: Benchmarking Individual-Level Privacy Reasoning in Vision-Language Models (ICML 2026 · similarity 0.85)
- The Emotional Baby Is Truly Deadly: Does Your Multimodal Large Reasoning Model Have Emotional Flattery Towards Humans? (AAAI 2026 · similarity 0.83)
- EmoNet-Face: An Expert-Annotated Benchmark for Synthetic Emotion Recognition (NeurIPS 2025 · similarity 0.83)
- AffectGPT-RL: Revealing Roles of Reinforcement Learning in Open-Vocabulary Emotion Recognition (NeurIPS 2026 · similarity 0.83)
- OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition (ICML 2025 · similarity 0.83)
- MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
