Research map / Graph learning / GNN architectures
Schur's Positive-Definite Network: Deep Learning in the SPD cone with structure
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SpodNet, a neural layer that maps SPD matrices to SPD matrices via sequential Schur-complement column-row updates, enabling jointly SPD and sparse outputs for sparse precision matrix estimation.
Problem
Learning neural networks whose outputs are guaranteed symmetric positive-definite (SPD) while also satisfying additional structural constraints such as elementwise sparsity, which existing learning-based and Riemannian methods cannot jointly enforce.
Approach
Update SPD matrices one column-row pair at a time: any learned off-diagonal column is accepted, and the diagonal entry is set to a positive slack plus the Schur complement term, which guarantees SPD output for any network choice; soft-thresholding inside the networks gives exact sparsity.
Key results
- On synthetic data with p=100, n=100 and weakly sparse ground truth, traditional methods (GLasso, Ledoit-Wolf, OAS) reach around 90% NMSE, while SpodNet variants substantially outperform them, especially in low-sample regimes.
- In the lower-dimensional p=20 setting, the UBG model achieves an F1 score above 0.75 for support recovery.
- On the Animals dataset, UBG reaches modularity 0.78, above GLasso (0.61) and GLAD (0.72), and below EGFM (0.86).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UBG instead of GLasso | graph learning on Animals · Animals | modularity: 0.61 -> 0.78 | Helped |
| Use UBG instead of GLAD | graph learning on Animals · Animals | modularity: 0.72 -> 0.78 | Helped |
| Use UBG instead of EGFM | graph learning on Animals · Animals | modularity: 0.86 -> 0.78 | Hurt |
| Use learning-based methods (UBG, PNP, or E2E) instead of traditional estimators | sparse precision matrix recovery · synthetic test data | NMSE | Helped |
| Use SpodNet models instead of GLasso as sample size grows | sparse precision matrix recovery · synthetic strongly sparse settings | F1 score: up to 25% improvement | Helped |
| Use UBG for support recovery in the lower-dimensional setting | sparse precision matrix recovery · synthetic p=20 | F1 score: over 0.75 | Helped |
| Use SpodNet models in the sample-deficient regime | sparse precision matrix recovery · synthetic n <= p | NMSE | Helped |
| Multiply the learned threshold parameter by 0.1 (ablation) | PNP · synthetic sparse precision matrix recovery | convergence: 0.1 | Helped |
| Multiply the learned threshold parameter by 0.1 (ablation) | E2E · synthetic sparse precision matrix recovery | convergence: 0.1 | Helped |
| Scale the final-layer preactivation to control the column update magnitude (ablation) | SpodNet · training stability | conditioning stability: zeta = 1 | Helped |
| Remove the update normalization (ablation) | SpodNet · training | conditioning stability | Hurt |
Surprising findings
- GLAD's Z matrix, which is sparse, is almost never SPD in strongly sparse p=100 settings, and projecting it onto the SPD cone destroys sparsity.
- Without stabilization, the smallest eigenvalue of SpodNet's Theta can approach zero during training, causing instability.
Limitations
- Computational cost of full layer update is O(p^3) and the framework's cost is a stated future-work item.
- No formal convergence guarantees or theoretical analysis of expressivity or eigenvalue dynamics.
- Experiments are limited to sparse precision matrix estimation with one-layer (K=1) models and small MLPs.
Experimental setup
Datasets: synthetic sparse SPD matrices from sklearn make_sparse_spd_matrix Animals dataset
Benchmarks: synthetic sparse precision matrix recovery Animals dataset graph learning
Metrics: NMSE F1 score for support recovery modularity
Baselines: GLasso Ledoit-Wolf OAS GLAD EGFM DeepGraph
Builds on: Graphical Lasso GLAD Graphical ISTA Plug-and-Play priors algorithm unrolling
Closest papers by meaning
- A Riemannian Network for SPD Matrix Learning (AAAI 2017 · similarity 0.87)
- Riemannian Local Mechanism for SPD Neural Networks (AAAI 2023 · similarity 0.86)
- Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices (NeurIPS 2024 · similarity 0.85)
- Implicit bias of SGD in $L_2$-regularized linear DNNs: One-way jumps from high to low rank (ICLR 2024 · similarity 0.84)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.84)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.