Research map / Label-efficient and robust learning / Recognition and long-tail

ProtoGate: Prototype-based Neural Networks with Global-to-local Feature Selection for Tabular Biomedical Data

ICML 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Prototype and concept dictionary models · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ProtoGate, a prototype-based neural model that combines global-to-local soft feature selection with a non-trainable KNN-style predictor to improve accuracy, sparsity and fidelity of feature selection on HDLSS tabular biomedical data.

Problem

Feature selection and prediction on high-dimensional low-sample-size (HDLSS) tabular biomedical data, where existing local (instance-wise) feature selection methods miss globally important features and suffer from co-adaptation between selector and predictor.

Approach

Sequentially applies l1-regularised global selection and l0-regularised local selection via a gating network, then makes predictions with a non-parametric differentiable KNN over masked prototypes; the predictor has no trainable parameters, so only the gate is learned, which mitigates co-adaptation and aligns with the clustering assumption. A hybrid sorting strategy (NeuralSort in training, QuickS

Key results

• ProtoGate achieves the best average rank (2.00) across seven HDLSS datasets, versus 5.86 for the best local baseline REAL-X.
• ProtoGate reaches 83.95% balanced accuracy on colon and 95.96% on meta-pam, outperforming LSPIN and other local methods on most datasets.
• Hybrid sorting reportedly cuts inference time by almost half while keeping predictive performance identical to using only differentiable sorting.

Measured effects reported in the paper

Surprising findings

• L2X reaches 96% accuracy on MNIST using only one pixel, illustrating low-fidelity selection with high accuracy.
• Simple Lasso and MLP outperform many local feature selection methods on HDLSS datasets.

Limitations

• Prototype-based prediction may perform poorly when informative features rely on even functions (e.g., absolute value), as shown on the Syn3 dataset, violating the clustering assumption.
• Designed for the HDLSS regime; generalisation to non-HDLSS datasets is shown only briefly on four datasets.

Experimental setup

Datasets: colon lung meta-dr meta-pam prostate tcga-2y toxicity 100-plants-texture cnae-9 mfeat-fourier vehicle Syn1

Benchmarks: colon lung meta-dr meta-pam prostate tcga-2y toxicity 100-plants-texture cnae-9 mfeat-fourier vehicle Syn1

Metrics: balanced accuracy average rank ADTM F1 of selected features proportion of features selected per sample runtime model size

Baselines: Ridge SVM KNN MLP Lasso Random Forest XGBoost CatBoost LightGBM STG TabNet L2X

Builds on: LSPIN STG REAL-X NeuralSort L2X

Closest papers by meaning

• SAND: One-Shot Feature Selection with Additive Noise Distortion (ICML 2025 · similarity 0.83)
• Contextual Feature Selection with Conditional Stochastic Gates (ICML 2024 · similarity 0.81)
• PTaRL: Prototype-based Tabular Representation Learning via Space Calibration (ICLR 2024 · similarity 0.81)
• TabPrep: Closing the Feature Engineering Gap in Tabular Benchmarks (NeurIPS 2026 · similarity 0.81)
• A Data-Centric Perspective on Evaluating Machine Learning Models for Tabular Data (NeurIPS 2024 · similarity 0.81)
• Small Models are LLM Knowledge Triggers for Medical Tabular Prediction (ICLR 2025 · similarity 0.81)