Research map / Architectures and efficiency / Attention and transformers
Neural Embeddings for kNN Search in Biological Sequence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bio-kNN, which clusters all training triplets by EMD-based cell distance distributions, samples triplets per cluster, and uses a CNN backbone with per-cluster MLP heads plus CNN features for biological sequence kNN search.
Problem
Neural distance embeddings for biological sequence kNN search (approximating Needleman-Wunsch distance) are trained with triplet loss on a small, intuitively chosen subset of triplets, so they recognize only part of the distance orders; naively adding triplets to SOTA networks does not help.
Approach
Partition the full triplet selection space into cells, cluster cells by their EMD-based distance distributions, sample one triplet per cluster per anchor via a batch-wise assignment strategy, and train a CNN backbone with one MLP head per cluster, concatenating CNN features into the final embedding.
Key results
- On Uniprot, Bio-kNN reaches HR@1 54.00 vs 47.70 for CNNED (state-of-the-art baseline), a gain of 4.9 points.
- On Uniref, Bio-kNN reaches HR@10 32.51 vs 28.55 for CNNED, a gain of 3.96 points.
- Ablations show removing the multi-head network or CNN features reduces performance on both datasets (e.g., Uniprot HR@1 50.40 without CNN features vs 54.00 full).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bio-kNN instead of the strongest reported baseline | Biological sequence kNN search · Uniprot | HR@1: 49.10 -> 54.00 | Helped |
| Use Bio-kNN instead of the strongest reported baseline | Biological sequence kNN search · Uniprot | HR@10: 34.58 -> 42.69 | Helped |
| Use Bio-kNN instead of the strongest reported baseline | Biological sequence kNN search · Uniref | HR@1: 35.13 -> 37.60 | Helped |
| Use Bio-kNN instead of the strongest reported baseline | Biological sequence kNN search · Uniref | HR@10: 28.55 -> 32.51 | Helped |
| Use clustered triplet selection rather than 1 cluster (ablation) | Bio-kNN-Base · Uniprot | HR@1: 48.30 -> 50.50 | Helped |
| Use clustered triplet selection rather than 1 cluster (ablation) | Bio-kNN-Base · Uniref | HR@1: 28.30 -> 33.90 | Helped |
| Add the multi-head network rather than using a single head (ablation) | Bio-kNN · Uniprot | HR@1: 53.20 -> 54.00 | Helped |
| Add concatenated CNN features (ablation) | Bio-kNN · Uniprot | HR@1: 50.40 -> 54.00 | Helped |
| Replace HAC segmentation with average-based segmentation (ablation) | Bio-kNN · Uniprot | HR@1: 54.00 -> 52.03 | Hurt |
| Add the multi-head network rather than using a single head (ablation) | Bio-kNN · Uniref | HR@1: 35.63 -> 37.60 | Helped |
| Add concatenated CNN features (ablation) | Bio-kNN · Uniref | HR@1: 33.67 -> 37.60 | Helped |
| Replace HAC segmentation with average-based segmentation (ablation) | Bio-kNN · Uniref | HR@1: 37.60 -> 35.43 | Hurt |
Surprising findings
- Naively training on more triplets (Method-5, combining several selection regions) under the CNNED pipeline did not improve test-set distance-order recognition.
- Regions outside the training selection still benefit from training, a radiation effect whose magnitude varies by position.
Limitations
- Only one intuitive property (distance distribution) used for cell clustering; other properties not explored.
- Training set limited to 1,000 sequences per dataset in experiments.
Experimental setup
Datasets: Uniprot Uniref
Benchmarks: Uniprot Uniref
Metrics: HR@1 HR@5 HR@10 HR@50 Top-1 Recall
Baselines: AsMac GRU CNNED NeuroSEED Global Transformer NeuroSEED Local Transformer
Builds on: CNNED GRU triplet selection triplet loss Needleman-Wunsch
Closest papers by meaning
- Levenshtein Distance Embedding with Poisson Regression for DNA Storage (AAAI 2024 · similarity 0.85)
- Hyperbolic Genome Embeddings (ICLR 2025 · similarity 0.83)
- Training Compute-Optimal Protein Language Models (NeurIPS 2024 · similarity 0.83)
- Revisiting Convolution Architecture in the Realm of DNA Foundation Models (ICLR 2025 · similarity 0.83)
- Clustering for Protein Representation Learning (CVPR 2024 · similarity 0.82)
- What DNA Foundation Models Learn Beyond Sequence Composition (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.