Research map / Video / Video segmentation and tracking
DepMGNN: Matrixial Graph Neural Network for Video-based Automatic Depression Assessment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Matrixial Graph and MGNN, a GNN whose nodes hold 2D feature maps and whose edges are matrix-valued, to model variable-length face videos end-to-end for depression assessment, reaching SOTA on AVEC2013/2014.
Problem
Video-based automatic depression assessment from long, variable-length face videos; existing methods resample, split into segments, or compress frames into vectors, losing spatio-temporal information.
Approach
A graph whose nodes are matrices (frame-level 2D feature maps at multiple spatial scales) with spatial and temporal edges, plus a GNN that learns matrix-valued edge features for message passing, so variable-length videos become graphs with varying node counts without resampling or vectorization.
Key results
- Achieves RMSE 6.62 and MAE 5.45 on AVEC 2014 (ResNet-50 variant, 7.08 RMSE on AVEC 2013 for ResNet-50 and 7.26 for SENet-50), state of the art on both datasets.
- Relative RMSE improvements of 2.5% on AVEC 2013 and 5.2% on AVEC 2014 over previous SOTA.
- Replacing MGNN with LSTM, GRU, Conv3D, ViT or Timesformer yields RMSE 8.10-8.78 on AVEC 2014, versus 6.62 for MGNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DepMGNN with ResNet-50 instead of the previous best RMSE baseline | AVEC 2013 video depression assessment · AVEC 2013 test | RMSE: 7.26 -> 7.08 | Helped |
| Fuse Freeform and Northwind predictions | DepMGNN · AVEC 2014 test | RMSE: 6.62 -> 6.28 | Helped |
| Replace vector-based GAT with MGNN on matrixial graph features (ablation) | MGL graph model · AVEC 2014 | RMSE: 8.01 -> 6.62 | Helped |
| Replace binary-value edges with matrixial edge features (ablation) | MGNN · AVEC 2014 | RMSE: 7.54 -> 6.62 | Helped |
| Include both spatial and temporal interactions (ablation) | MGNN · AVEC 2014 | RMSE: 7.49 -> 6.62 | Helped |
| Replace MGNN with LSTM (ablation) | DepMGNN · AVEC 2014 | RMSE: 6.62 -> 8.55 | Hurt |
| Replace MGNN with GRU (ablation) | DepMGNN · AVEC 2014 | RMSE: 6.62 -> 8.40 | Hurt |
| Replace MGNN with Conv3D (ablation) | DepMGNN · AVEC 2014 | RMSE: 6.62 -> 8.78 | Hurt |
| Replace MGNN with ViT (ablation) | DepMGNN · AVEC 2014 | RMSE: 6.62 -> 8.54 | Hurt |
| Replace MGNN with Timesformer (ablation) | DepMGNN · AVEC 2014 | RMSE: 6.62 -> 8.10 | Hurt |
| Add more spatial scales to the graph (ablation) | MGNN · AVEC 2014 | RMSE: 9.25 -> 6.62 | Helped |
| Use all four spatial scales instead of S1+S2+S3 (ablation) | MGNN · AVEC 2014 | RMSE: 8.07 -> 6.62 | Helped |
Surprising findings
- Reducing matrixial nodes to vectors and using standard GAT or GatedGCN still improves over the backbone, but matrix-valued edges give substantially larger gains than binary-valued edges.
Limitations
- Increased model complexity compared with prior approaches.
- Only evaluated on two AVEC depression datasets in the main text.
Experimental setup
Datasets: AVEC 2013 AVEC 2014
Benchmarks: AVEC 2013 AVEC 2014
Metrics: RMSE MAE PCC CCC
Baselines: Xu et al. 2024 Pan et al. 2023 Uddin et al. 2022 de Melo et al. 2020 Song et al. 2022 GAT GatedGCN LSTM GRU Conv3D ViT Timesformer
Builds on: GAT GatedGCN BMC loss ResNet
Closest papers by meaning
- Explainable Depression Assessment from Face Videos by Weakly Supervised Learning (AAAI 2026 · similarity 0.90)
- Dep-MAP: A Multi-level Alignment Framework with Semantic Prototypes for Video-based Automatic Depression Assessment (AAAI 2026 · similarity 0.89)
- Using Graph Representation Learning with Schema Encoders to Measure the Severity of Depressive Symptoms (ICLR 2022 · similarity 0.87)
- Personality-guided Public-Private Domain Disentangled Hypergraph-Former Network for Multimodal Depression Detection (AAAI 2026 · similarity 0.85)
- Depression Detection via Capsule Networks with Contrastive Learning (AAAI 2024 · similarity 0.82)
- Facial-R1: Aligning Reasoning and Recognition for Facial Emotion Analysis (AAAI 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.