Research map / Learning theory and algorithms / Prediction and decision losses

Explainable Neural Networks with Guarantee: A Sparse Estimation Approach

AAAI 2025 · Learning theory and algorithms › Prediction and decision losses · approach: Theory of neural network sensitivity · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SparXnet, a neural network that softmax-selects a sparse set of input features, applies a trainable 1-D neural function to each, and combines them linearly, with Lipschitz-based sample-complexity bounds and experiments on synthetic and six tabular datasets.

Problem

Interpretable neural networks that perform feature selection and nonlinear model estimation jointly, with generalization guarantees that do not depend on the number of network parameters or on the total number of input features.

Approach

Each first-layer neuron applies a temperature-controlled softmax over input weights so that it saturates to a single feature; a separate 1-D neural network transforms each selected feature, and the outputs are linearly combined. The Lipschitz-function class is then used to bound sample complexity independently of network size.

Key results

• On six real tabular datasets, SparXnet's average out-of-sample AUC is among the top three models on every dataset (e.g., 0.899 on Adult vs XGBoost 0.924; 0.910 on Credit Risk vs XGBoost 0.949).
• On the single-variable synthetic task with 2-5 noisy features, SparXnet's test MSE (about 0.005) is far below FCN and Lasso (about 0.12), and it recovers the true feature in all experiments.
• Theory gives a sample complexity of O(KL^2 log^3(...)) that is linear in the number of selected features K and only logarithmic in the input dimension d, with no dependence on network parameter count.

Measured effects reported in the paper

Surprising findings

• On the Heart Disease dataset SparXnet (AUC 0.853) is well below logistic regression (0.945) and XGBoost (0.950), and on Breast Cancer it is below logistic regression and XGBoost as well.
• Under the multi-variable synthetic setting SparXnet is only inferior to FCN when all features are used, and at eight selected features it slightly lags Lasso in recovery rate.

Limitations

• Softmax may not saturate, so some first-layer nodes mix multiple features, reducing interpretability.
• The final linear combination limits interpretability relative to rule-based methods, and transformation functions may be non-smooth.
• Results on some real datasets (Heart Disease, Breast Cancer) trail strong baselines such as XGBoost and logistic regression.

Experimental setup

Datasets: Heart Disease (UCI) Adult Income Breast Cancer Wisconsin Recidivism (COMPAS) Customer Churn Credit Bureau / Credit Risk

Benchmarks: Adult Income Breast Cancer Wisconsin Credit Risk Customer Churn Heart Disease Recidivism synthetic single-variable regression synthetic multi-variable regression

Metrics: AUC MSE RMSE true feature recovery rate

Baselines: fully connected neural network Lasso regression Ridge regression decision tree logistic regression NAM XGBoost

Builds on: neural additive model lasso projection pursuit regression generalized additive models

Closest papers by meaning

• Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations (ICML 2025 · similarity 0.90)
• Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (ICLR 2025 · similarity 0.88)
• Explain Yourself, Briefly! Self-Explaining Neural Networks with Concise Sufficient Reasons (ICLR 2025 · similarity 0.88)
• From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit (NeurIPS 2025 · similarity 0.88)
• Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants (NeurIPS 2026 · similarity 0.87)
• Q-SENN: Quantized Self-Explaining Neural Networks (AAAI 2024 · similarity 0.87)