Research map / Architectures and efficiency / RNNs and normalization
HORSE: Hierarchical Representation for Large-Scale Neural Subset Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HORSE, an attention-based hierarchical set encoder for neural subset selection that satisfies a defined Identity Property and permutation invariance, enabling chunked processing of large ground sets.
Problem
Neural subset selection needs set functions that capture interactions between the selected subset and the full ground set, while scaling to supersets too large to fit in memory.
Approach
Partition the ground set into chunks with learned slot embeddings, use a normalized subset-based attention to encode each chunk, and aggregate chunk encodings so the representation identifies the ground set regardless of partitioning (Identity Property), then concatenate with the subset encoding and feed an MLP.
Key results
- HORSE achieves 60.2 MJC on Two-Moons vs 58.5 for EquiVSet and 91.8 on Gaussian Mixture vs 90.9 for INSET.
- HORSE is best on 11 of 12 Amazon product categories and reaches 43.2 MJC on PDBBind-2 vs 37.1 for INSET.
- In large-set experiments (up to 1000 elements, partitioned into chunks), HORSE outperforms INSET and Set Transformer, with the paper claiming a margin of up to 20%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HORSE instead of the strongest baseline | Neural subset selection · Synthetic datasets: Two Moons and Gaussian Mixture | MJC: 58.5 ± 0.3 -> 60.2 ± 0.5; 90.9 ± 0.2 -> 91.8 ± 0.2 | Helped |
| Use HORSE instead of the strongest baseline | Product recommendation · Amazon baby registry categories | MJC: 80.8±1.2 -> 83.2±1.3; 88.3±0.7 -> 88.0±0.8 | Mixed |
| Use HORSE instead of the strongest baseline | Compound selection · PDBBind, BindingDB, PDBBind-2, BindingDB-2 | MJC: 93.5±0.8 -> 94.1 ± 0.7; 19.8±0.5 -> 21.3 ± 0.5 | Helped |
| Increase the Monte-Carlo sample number k from 2 to 8 (ablation) | HORSE product recommendation · Media | MJC: 63.1 ± 1.2 -> 66.8 ± 1.3 | Helped |
| Increase the Monte-Carlo sample number k from 2 to 10 (ablation) | HORSE product recommendation · Safety | MJC: 24.8 ± 1.3 -> 27.7 ± 0.8 | Helped |
| Increase the Monte-Carlo sample number k from 8 to 10 (ablation) | HORSE product recommendation · Media | MJC: 66.8 ± 1.3 -> 66.2 ± 0.9 | Hurt |
Surprising findings
- EquiVSet and INSET sometimes fall below DeepSet or Set Transformer on product categories such as Safety, Carseats and Furniture.
Limitations
- Large-scale support is only a theoretical partitioning framework on synthetic distributed setting, not real distributed experiments.
- Improvements on PDBBind and BindingDB single-filter tasks are marginal.
- Requires an EBM-based variational approximation with Monte-Carlo sampled subsets, adding training complexity.
Experimental setup
Datasets: Two-Moons Gaussian Mixture Amazon baby registry Double MNIST CelebA F-MNIST CIFAR-10 PDBBind BindingDB
Benchmarks: Two-Moons Gaussian Mixture Amazon baby registry product recommendation Double MNIST CelebA F-MNIST CIFAR-10 PDBBind BindingDB
Metrics: mean Jaccard coefficient (MJC)
Baselines: Random PGM DeepSet Set Transformer EquiVSet INSET
Builds on: EquiVSet INSET Set Transformer DeepSets Set Slot Attention
Closest papers by meaning
- Enhancing Neural Subset Selection: Integrating Background Information into Set Representations (ICLR 2024 · similarity 0.92)
- Efficient Biological Data Acquisition through Inference Set Design (ICLR 2025 · similarity 0.85)
- Balancing Feature Similarity and Label Variability for Optimal Size-Aware One-shot Subset Selection (ICML 2024 · similarity 0.84)
- Customized Subgraph Selection and Encoding for Drug-drug Interaction Prediction (NeurIPS 2024 · similarity 0.84)
- Foundation Model Insights and a Multi-Model Approach for Superior Fine-Grained One-shot Subset Selection (ICML 2025 · similarity 0.83)
- Most Influential Subset Selection: Challenges, Promises, and Beyond (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.