AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Sequential Attention for Feature Selection

Taisuke Yasuda, Mohammadhossein Bateni, Lin Chen, Matthew Fahrbach, Gang Fu, Vahab Mirrokni

ICLR 2023 · Learning theory and algorithms › Prediction and decision losses · approach: Data attribution and recourse · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Select features adaptively over multiple rounds (fewer features per round) instead of selecting many at once (ablation)Sequential Attention · ISOLET, 64 featuresaccuracy: 0.934 (1 per round) vs 0.782 (64 per round)Improved
Select features adaptively (fewer per round) instead of in large batches (ablation)Sequential Attention · MNIST, 64 featuresaccuracy: 0.963 (1 per round) vs 0.932 (64 per round)Improved
Adaptivity (more rounds) on mice protein (ablation)Sequential Attention · Mice Protein, 64 featuresaccuracy: 0.990 vs 0.990 (differences within noise)No clear effect
Replace softmax overparameterization with other Hadamard parameterizations (l1, l2, normalized variants) (ablation)Sequential Attention · Small-scale benchmarks (MNIST, ISOLET, Activity, etc.)accuracy: Mostly within ~0.01 of softmax; e.g., MNIST 0.957-0.958 vs softmax 0.958No clear effect
Replace softmax with unnormalized l1 |w| Hadamard parameterization (ablation)Sequential Attention · ISOLETaccuracy: 0.894 vs 0.920 (softmax)Hurt
Use normalized l1 instead of softmax overparameterization (ablation)Sequential Attention · ISOLET and Activityaccuracy: ISOLET 0.921 vs 0.920; Activity 0.933 vs 0.922No clear effect
Sequential Attention vs Group LASSO / Sequential LASSO on small-scale benchmarksSequential Attention · Small-scale benchmark suite (k=50)accuracy: SA best on ISOLET (0.920 vs GL 0.906) but OMP higher on Mice Protein (0.994 vs 0.993) and GL higher on Activity (0.933 vMixed
Sequential Attention vs LASSO-based methods on Criteo at larger kSequential Attention · Criteo large, k>=15 AUCAUC: SA 0.72659 vs GL(1e-1) 0.72392 at k=15; GL(1e-4) 0.73512 vs SA 0.73495 at k=35Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy