Research map / Learning theory and algorithms / Clustering
Heterogeneous Sufficient Dimension Reduction and Subspace Clustering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes mixPFC, a mixture of principal fitted components model that jointly performs supervised subspace clustering, subspace estimation and variable selection via a group-Lasso penalized EM algorithm with non-asymptotic convergence guarantees.
Problem
Sufficient dimension reduction assumes a homogeneous population, and subspace clustering ignores the response and needs separation conditions; there is no method that jointly clusters heterogeneous data and estimates regression-relevant subspaces in high dimensions.
Approach
Combine model-based SDR (principal fitted components) with a latent-cluster mixture so each cluster has its own central subspace; estimate subspaces by a convex group-Lasso problem over a roughly p-dimensional parameter, avoiding p x p inversions, within an EM algorithm.
Key results
- On simulations (K=2, p=1000), mixPFC keeps clustering error roughly around 10% or below, versus about 40-48% for most subspace clustering baselines in many settings; mixPFC-ISO gets the lowest error for M2.
- On CCLE Nutlin-3, mixPFC PMSE is 8.9 (x100) versus 17.6-18.8 for homogeneous SDR and Lasso baselines.
- On AIS data, mixPFC recovers the sex subpopulations with an error rate of 0.074.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mixPFC instead of Lasso | CCLE drug-response prediction · Nutlin-3 | PMSE ×100: 17.6 (0.3) -> 8.9(0.4) | Helped |
| Use mixPFC instead of Lasso | CCLE drug-response prediction · AZD6244 | PMSE ×100: 77.6 (1.5) -> 45.6 (1.7) | Helped |
| Use mixPFC instead of K-means on X ◦ Y | mixPFC · M1, K=2, Δ=0.1Ip | clustering error rate: 20.2 (0.3) -> 3.4 (0.7) | Helped |
| Use mixPFC instead of LSA | mixPFC · M2, K=2, Δ=0.1Ip | clustering error rate: 4.9 (0.1) -> 16.4 (2.0) | Hurt |
| Use mixPFC instead of K-means on X ◦ Y | mixPFC · M3, K=2, Δ=0.1Ip | clustering error rate: 22.3 (0.3) -> 3.6 (1.2) | Helped |
| Use mixPFC instead of SSC | mixPFC · M4, K=3, Δ=0.1Ip | clustering error rate: 6.5 (0.1) -> 0 (0) | Helped |
| Use mixPFC instead of SSC | mixPFC · M4, K=5, Δ=0.1Ip | clustering error rate: 13.8 (0.2) -> 0.2 (0.2) | Helped |
Surprising findings
- Subspace clustering methods match or beat mixPFC in some orthogonal-subspace settings with small noise (K=2, Delta=0.1Ip).
Limitations
- Theory is developed only for K=2 clusters with known covariance sigma^2 and fixed known fitting function f(Y).
- Multi-cluster high-dimensional EM theory is an open problem; K=5 clustering errors rise to around 15%.
- Initialization and selection of K are needed and rely on heuristics described in the appendix.
Experimental setup
Datasets: AIS dataset CCLE dataset synthetic data from mixPFC model
Benchmarks: AIS dataset CCLE dataset
Metrics: clustering error rate prediction mean squared error number of selected variables
Baselines: RANSAC SSC LSA LRSC GPCA K-means hierarchical clustering LassoSIR SEAS-SIR SEAS-PFC Lasso
Builds on: principal fitted components probabilistic PCA SEAS-PFC LassoSIR mixture of linear regression
Closest papers by meaning
- Subspace clustering in high-dimensions: Phase transitions & Statistical-to-Computational gap (NeurIPS 2022 · similarity 0.88)
- Stochastic Sparse Subspace Clustering (CVPR 2020 · similarity 0.88)
- Mixed-Features Vectors and Subspace Splitting (ICLR 2021 · similarity 0.87)
- Diverse Influence Component Analysis: A Geometric Approach to Nonlinear Mixture Identifiability (NeurIPS 2025 · similarity 0.85)
- Efficient Sparse PCA via Block-Diagonalization (ICLR 2025 · similarity 0.85)
- Achieving Optimal Clustering in Gaussian Mixture Models with Anisotropic Covariance Structures (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.