Research map / Label-efficient and robust learning / Recognition and long-tail
QCS:Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Quadruplet Cross Similarity (QCS), a four-branch centrally symmetric network with a Cross Similarity Attention module and contrastive residual distillation, that refines facial expression features by contrasting same- and different-class image pairs during training and keeps only one base branch at inference.
Problem
Facial expression recognition suffers from unlabeled redundant features mixed with labeled discriminative features, and standard ViT scaled dot-product attention does not give direct feedback when applied to pairs of different images.
Approach
Cross Similarity Attention computes Euclidean-distance-based similarity between query and key of two images and applies the resulting spatial weights directly to the branch's own values; a four-branch closed-loop (anchor, positive, two negatives) network with same-class similarity S plus different-class distance D fused via a learnable tanh-gated term, and residual distillation of the cross classi
Key results
- QCS achieves 92.50% on RAF-DB, 91.41% on FERPlus, and 67.94% on AffectNet-7 with IR50 backbone, and 93.02% / 91.85% on RAF-DB / FERPlus when pretrained on AffectNet-8.
- Ablation on RAF-DB: CSA with element-wise addition of GAP outputs reaches 92.14 vs. the 92.05 baseline; the SD fusion of same-class similarity and different-class distance reaches 92.47 versus 92.18 for S alone.
- QCS reports 64.30% on AffectNet-8, while DCS reports 64.40%, which is the best AffectNet-8 result in the table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SDPA with CSA in the cross-attention module (ablation) | DCS · RAF-DB and FERPlus | accuracy: 91.95 -> 92.14; 90.93 -> 91.02 | Helped |
| add CSA without a residual connection (ablation) | IR50 + ViT baseline · RAF-DB and FERPlus | accuracy: 92.05 -> 91.98; 91.06 -> 90.83 | Hurt |
| add element-wise addition followed by GAP (ablation) | DCS with CSA · RAF-DB and FERPlus | accuracy: 91.98 -> 92.14; 90.83 -> 91.02 | Mixed |
| add bilinear pooling (ablation) | DCS with CSA · RAF-DB and FERPlus | accuracy: 91.98 -> 92.24; 90.83 -> 91.25 | Helped |
| add ViT connection to align cross and base classifiers (ablation) | DCS with CSA · RAF-DB and FERPlus | accuracy: 91.98 -> 92.57; 90.83 -> 91.22 | Mixed |
| add KL divergence loss (ablation) | DCS with CSA and bilinear pooling · RAF-DB and FERPlus | accuracy: 92.24 -> 92.21; 91.25 -> 90.93 | Hurt |
| retain only intra-class similarity matrix S rather than fused SD (ablation) | QCS · RAF-DB and FERPlus | accuracy: 92.47 -> 92.18; 91.21 -> 91.12 | Hurt |
| retain only inter-class distance matrix D rather than fused SD (ablation) | QCS · RAF-DB and FERPlus | accuracy: 92.47 -> 92.11; 91.21 -> 90.96 | Hurt |
| fuse intra-class similarity S and inter-class distance D rather than use either alone (ablation) | QCS · RAF-DB and FERPlus | accuracy: 92.18 -> 92.47; 91.12 -> 91.21 | Helped |
| use QCS instead of DCS | FER · RAF-DB, FERPlus, AffectNet-7, and AffectNet-8 | accuracy: 92.57 -> 92.50; 91.41 -> 91.41; 67.66 -> 67.94; 64.40 -> 64.30 | Mixed |
| pretrain QCS on AffectNet-8 | QCS · RAF-DB and FERPlus | accuracy: 92.50 -> 93.02; 91.41 -> 91.85 | Helped |
Surprising findings
- Naively adding KL or MSE distillation between cross and base classifiers reduced accuracy, which the authors attribute to overfitting of the cross module.
- The four-branch QCS did not significantly outperform the two-branch DCS variant.
Limitations
- The cross module appears to overfit, and gains depend on dataset-specific augmentation and sampling strategies.
- Transferring cross-module gains to the inference base network remains an open challenge.
- Training a stable triplet-style network without the four-branch design failed in the authors' attempts.
Experimental setup
Datasets: RAF-DB FERPlus AffectNet MS-Celeb-1M
Benchmarks: RAF-DB FERPlus AffectNet-7 AffectNet-8
Metrics: accuracy
Baselines: POSTER++ POSTER S2D APViT TAN TransFER DMUE RUL
Builds on: Vision Transformer (ViT) Non-local block POSTER++ Triplet loss Sharpness Aware Minimization
Closest papers by meaning
- When Facial Expression Recognition Meets Few-Shot Learning: A Joint and Alternate Learning Framework (AAAI 2022 · similarity 0.86)
- Feature Decomposition and Reconstruction Learning for Effective Facial Expression Recognition (CVPR 2021 · similarity 0.86)
- Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis (AAAI 2026 · similarity 0.83)
- Hypergraph-Guided Disentangled Spectrum Transformer Networks for Near-Infrared Facial Expression Recognition (AAAI 2024 · similarity 0.82)
- Open-Set Facial Expression Recognition (AAAI 2024 · similarity 0.82)
- FAIR-FER: A Latent Alignment Approach for Mitigating Bias in Facial Expression Recognition (Student Abstract) (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.