Research map / Label-efficient and robust learning / Recognition and long-tail
Selective Deep Autoencoder for Unsupervised Feature Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Selective Deep Autoencoder (SDAE), a deep autoencoder with a non-negative L1-regularized Selective Layer that learns a feature subset reconstructing latent encodings for unsupervised feature selection without fixing k in advance.
Problem
Unsupervised feature selection on high-dimensional data: choosing a compact, globally representative feature subset without labels, and without fixing the number of features in advance.
Approach
Pre-train a deep autoencoder for reconstruction, then train a Selective Layer (diagonal non-negative weights with L1 sparsity) plus the encoder to reproduce the pre-trained latent encodings from the weighted input, so the number of selected features is learned rather than set a priori.
Key results
- On the simulated data, SDAE automatically selected 4 features with real reconstruction MSE 2.32e-3, versus 0.4283 and 0.1027 for FAE with k=4 and k=5.
- On Mice Protein, SDAE reached 94.4% classification accuracy versus 87.8% for the best baseline (FAE).
- SDAE reconstruction errors were among the top three smallest among baselines on all three real datasets, though FAE and CAE were close on some metrics (e.g., MNIST accuracy 91.3% for SDAE vs 92.9% for FAE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDAE instead of FAE with k = 4 | Unsupervised feature selection on simulated data with all features · Simulated data, all features (features 1–28), test | real reconstruction MSE: 0.4283 -> 2.32e−3 | Helped |
| Use SDAE instead of FAE with k = 4 | Unsupervised feature selection on simulated data with all features · Simulated data, nonlinear features (features 17–28), test | nonlinear feature reconstruction MSE: 0.9951 -> 5.21e−3 | Helped |
| Use SDAE instead of FAE with k = 1 | Unsupervised feature selection on simulated data with linear features · Simulated data, linear features (features 1–16), test | real reconstruction MSE: 9.08e−6 -> 6.04e−5 | Mixed |
| Use SDAE instead of UDFS | Unsupervised feature selection · Mice Protein | linear reconstruction error (×10−3): 9 ± 6 -> 26 ± 2 | Hurt |
| Use SDAE instead of UDFS | Unsupervised feature selection · ISOLET | linear reconstruction error (×10−3): 19 ± 1 -> 16 ± 1 | Helped |
| Use SDAE instead of FAE | Unsupervised feature selection · MNIST | linear reconstruction error (×10−3): 19 ± 0 -> 19 ± 0 | No clear effect |
| Use SDAE instead of FAE | Unsupervised feature selection with downstream classification · Mice Protein | classification accuracy (%): 87.8 ± 7.3 -> 94.4 ± 1.4 | Helped |
| Use SDAE instead of FAE | Unsupervised feature selection with downstream classification · ISOLET | classification accuracy (%): 89.0 ± .6 -> 88.5 ± .6 | Hurt |
| Use SDAE instead of FAE | Unsupervised feature selection with downstream classification · MNIST | classification accuracy (%): 92.9 ± .7 -> 91.3 ± .3 | Hurt |
Surprising findings
- On MNIST and ISOLET, SDAE's classification accuracy was slightly below FAE, and on ISOLET and MNIST the baseline FAE scored higher on accuracy than SDAE.
- The authors argue that the magnitude of Selective Layer weights does not reflect feature importance, so selection uses only zero versus non-zero.
Limitations
- Number of selected features is controlled indirectly through the L1 coefficient tuned to approximate k for baselines, so comparisons with fixed-k methods are approximate.
- Simulation and real-data results are limited to three datasets and a single hand-designed simulation.
- SDAE does not uniformly beat baselines; accuracy on MNIST and ISOLET is slightly below FAE.
Experimental setup
Datasets: Simulated nonlinear dataset Mice Protein ISOLET MNIST
Benchmarks: Mice Protein ISOLET MNIST Simulated dataset
Metrics: linear reconstruction error (MSE) classification accuracy (extremely randomized trees) scaled and real reconstruction MSE
Baselines: LS AEFS UDFS MCFS AgnoS-S CAE FAE
Builds on: Fractal Autoencoders Concrete Autoencoders AgnoS-S Adam
Closest papers by meaning
- SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.85)
- Indirectly Parameterized Concrete Autoencoders (ICML 2024 · similarity 0.84)
- ScopeSAE: Model-Scope Feature Discovery with Interpretable Layer Selection (NeurIPS 2026 · similarity 0.83)
- A CLIP-Powered Framework for Robust and Generalizable Data Selection (ICLR 2025 · similarity 0.83)
- A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data (ICLR 2024 · similarity 0.82)
- PEAKS: Selecting Key Training Examples Incrementally via Prediction Error Anchored by Kernel Similarity (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.