Research map / Label-efficient and robust learning / Recognition and long-tail
Disentangling Interpretable Factors with Supervised Independent Subspace Principal Component Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes sisPCA, a linear multi-subspace PCA extension that uses supervision via HSIC and HSIC-based disentanglement penalties to learn independent, interpretable subspaces, solved by alternating eigendecomposition.
Problem
Learning high-dimensional representations whose subspaces align with known (supervised) and unknown attributes while staying disentangled; linear methods handle one space, while deep generative disentanglement models are hard to fit and not interpretable.
Approach
Extends supervised PCA to multiple subspaces, each supervised by a target kernel and penalized by HSIC (linear kernel for the linear variant) against other subspaces, yielding a closed-form alternating eigendecomposition with links to linear autoencoders and Zellner's g-prior regression.
Key results
- On breast cancer data, sisPCA raised the radius-subspace silhouette score for diagnosis to 0.516 versus 0.470 for sPCA while increasing subspace separation (Grassmann distance 2.710 vs 1.493).
- On the scRNA-seq infection task, sisPCA achieved infection silhouette 0.235 versus 0.207 for sPCA and 0.041 for supVAE, with Grassmann distance 4.824 vs 3.771 for sPCA.
- On TCGA DNAm, increasing lambda reduced HSIC-linear between aging and cancer subspaces from 189.7 (lambda=0) to 2.4e-5 (lambda=10).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add HSIC disentanglement regularization (λ = 1) (ablation) | sisPCA on TCGA DNAm · TCGA DNAm, age and cancer subspaces | Grassmann distance: 3.09 -> 4.97 | Helped |
| increase λ from 1 to 10 (ablation) | sisPCA on TCGA DNAm · TCGA DNAm, age subspace | maximum Spearman correlation with age: 0.286 -> 0.294 | Helped |
| increase λ from 1 to 10 (ablation) | sisPCA on TCGA DNAm · TCGA DNAm, cancer subspace | Silhouette score with cancer type: 0.107 -> 0.097 | Hurt |
| add HSIC disentanglement regularization | sisPCA on mouse malaria scRNA-seq · Mouse malaria scRNA-seq, infection and time subspaces | Grassmann distance: 3.771 -> 4.824 | Helped |
| use sisPCA rather than sPCA | infection subspace on mouse malaria scRNA-seq · Mouse malaria scRNA-seq, infection subspace | Silhouette - infection: 0.207 -> 0.235 | Helped |
| use sisPCA rather than hsVAE | infection subspace on mouse malaria scRNA-seq · Mouse malaria scRNA-seq, infection subspace | Silhouette - infection: -0.015 -> 0.235 | Helped |
| use sisPCA rather than sPCA | radius subspace on Breast Cancer Wisconsin · Breast Cancer Wisconsin, radius subspace | Silhouette score: 0.470 -> 0.516 | Helped |
| add disentanglement between radius and symmetry subspaces | sisPCA on Breast Cancer Wisconsin · Breast Cancer Wisconsin, radius and symmetry subspaces | Grassmann distance: 1.493 -> 2.710 | Helped |
| use sisPCA rather than hsVAE | time subspace on mouse malaria scRNA-seq · Mouse malaria scRNA-seq, time subspace | Silhouette - time point: 0.582 -> 0.355 | Hurt |
Surprising findings
- Supervised subspaces with a linear HSIC penalty can still show residual Gaussian-HSIC dependence, and the Gaussian-kernel variant (sisPCA-general) struggled to fully remove confounding signals.
- Stronger disentanglement regularization does not always improve representation quality because aging and tumorigenesis are coupled, causing information loss on cancer type.
Limitations
- Linear nature may miss nonlinear feature interactions; nonlinear extension not implemented.
- Linear-kernel HSIC does not guarantee statistical independence of subspaces.
- Identifiability issues when supervisions are similar or a subspace is unsupervised.
Experimental setup
Datasets: Kaggle Breast Cancer Wisconsin Data TCGA DNA methylation (450K, downsampled) mouse hepatocyte scRNA-seq Plasmodium liver stage
Benchmarks: Kaggle Breast Cancer Wisconsin Data TCGA DNA methylation pan-cancer mouse liver Plasmodium scRNA-seq
Metrics: silhouette score Geodesic Grassmann distance HSIC-linear HSIC-Gaussian maximum Spearman correlation with age
Baselines: PCA supervised PCA (sPCA) VAE supVAE hsVAE HCV
Builds on: supervised PCA Hilbert-Schmidt Independence Criterion contrastive PCA HCV
Closest papers by meaning
- Principle Component Trees and Their Persistent Homology (AAAI 2024 · similarity 0.85)
- Contrastive Functional Principal Component Analysis (AAAI 2025 · similarity 0.84)
- Heterogeneous Sufficient Dimension Reduction and Subspace Clustering (ICML 2025 · similarity 0.82)
- Interpretable Causal Representation Learning for Biological Data in the Pathway Space (ICLR 2025 · similarity 0.82)
- Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning (ICLR 2024 · similarity 0.82)
- Symb-xMIL: Symbolic Explanations for Multiple Instance Learning in Digital Pathology (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.