Research map / Label-efficient and robust learning / Recognition and long-tail
Open-Set Facial Expression Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes open-set facial expression recognition and reframes it as noisy-label detection, using cyclic two-model training with attention-map consistency on pseudo labels from a closed-set FER model.
Problem
Closed-set FER models are overconfident on unseen expression classes (e.g., compound expressions), and existing open-set recognition methods perform poorly because FER classes have small inter-class distances.
Approach
Small inter-class distance makes open-set FER pseudo labels spread across all known classes, resembling symmetric label noise; the method therefore treats open-set detection as noisy-label detection, using cyclic two-model co-teaching with a cyclical learning rate and attention map consistency, then separates samples by classification loss.
Key results
- On one-class open-set FER, the method reaches mean AUROC 0.909 on RAF-DB and 0.882 on FERPlus, versus 0.749 for the best prior method (PROSER) on RAF-DB.
- Ablation on RAF-DB (surprise open class): attention consistency raises AUROC from 0.517 to 0.885; adding cycle LR and cycle training reaches 0.918.
- On compound expressions the method reaches AUROC 0.771 versus 0.674 for DIAS; an online variant reaches mean AUROC 0.883.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed open-set FER method instead of the strongest baseline, PROSER | RAF-DB one-open-class detection · RAF-DB, one open class (mean) | AUROC: 0.749 -> 0.909 | Helped |
| Use the proposed open-set FER method instead of the strongest baseline, PROSER | RAF-DB one-open-class detection · RAF-DB, one open class (mean) | FPR@TPR95: 0.811 -> 0.547 | Helped |
| Use the proposed open-set FER method instead of the strongest baseline, PROSER | FERPlus one-open-class detection · FERPlus, one open class (mean) | AUROC: 0.739 -> 0.882 | Helped |
| Use the proposed open-set FER method instead of the strongest baseline, PROSER | Detection with multiple open classes · FER, two or three open classes (mean) | AUROC: 0.749 -> 0.876 | Helped |
| Use the proposed open-set FER method instead of the strongest baseline, EOW | RAF-DB compound-expression detection · RAF-DB compound expressions | AUROC: 0.679 -> 0.771 | Helped |
| Use the proposed open-set FER method instead of the strongest baseline, PROSER | AffectNet open-set detection · AffectNet, contempt open | AUROC: 0.610 -> 0.674 | Helped |
| Use online detection with the trained model instead of offline detection | Open-set FER · Seven open-class settings (mean) | AUROC: 0.909 -> 0.883 | Hurt |
| Add attention map consistency (ablation) | Open-set FER method · RAF-DB, Surprise open | AUROC: 0.517 -> 0.885 | Helped |
| Add cycle learning rate (ablation) | Open-set FER method with attention map consistency · RAF-DB, Surprise open | AUROC: 0.885 -> 0.912 | Helped |
| Add cycle training (ablation) | Open-set FER method with attention map consistency and cycle learning rate · RAF-DB, Surprise open | AUROC: 0.912 -> 0.918 | Helped |
| Concentrate open-set pseudo labels in the most similar close-set class instead of distributing them across classes (ablation) | Open-set FER method · RAF-DB, Surprise open | AUROC: 0.918 -> 0.803 | Hurt |
Surprising findings
- The softmax baseline scores near random (mean AUROC 0.497 on RAF-DB), since FER overconfidence makes open-set and close-set confidences overlap.
- Concentrating open-set pseudo labels on a single similar class (happiness) drops AUROC from 0.918 to 0.803, showing the spread of pseudo labels is informative.
Limitations
- Requires training from scratch on the test set, which is not directly online (online variant loses 2.6% AUROC).
- Performance drops on noisy datasets such as AffectNet (AUROC 0.674) and on compound expressions (0.771).
- Technical contribution is described by the authors as relatively small; relies on GMM-based clean sample selection with a fixed threshold of 0.5.
Experimental setup
Datasets: RAF-DB FERPlus AffectNet
Benchmarks: RAF-DB open-set FERPlus open-set AffectNet contempt open-set RAF-DB compound expressions
Metrics: AUROC FPR@TPR95
Baselines: MSP EOW PROSER DIAS
Builds on: CAM Gaussian Mixture Models loss modeling random erasing DIAS PROSER
Closest papers by meaning
- FERV39k: A Large-Scale Multi-Scene Dataset for Facial Expression Recognition in Videos (CVPR 2022 · similarity 0.89)
- Leave No Stone Unturned: Mine Extra Knowledge for Imbalanced Facial Expression Recognition (NeurIPS 2023 · similarity 0.88)
- Navigating Label Ambiguity for Facial Expression Recognition in the Wild (AAAI 2025 · similarity 0.88)
- FAIR-FER: A Latent Alignment Approach for Mitigating Bias in Facial Expression Recognition (Student Abstract) (AAAI 2024 · similarity 0.86)
- Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis (AAAI 2026 · similarity 0.85)
- Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.