AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Ramba: Selective State-Space Models for Relational Deep Learning

Yiming Liu, Chunyu Wei, Haozhe Lin, Fengjun Xiao, Junqi Zhang, Yunhai Wang, Yueguo Chen

ICML 2026 · Architectures and efficiency › Attention and transformers · approach: State space and recurrent sequence models

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Mamba backbone to sparse attention only model (ablation)Ramba · RelBench zero-shot regressionR2 (%): 17.5 -> 25.2Improved
add Mamba when sparse attention is disabled (ablation)Ramba · RelBench zero-shot classificationAUROC (%): 50.0 -> 68.5Improved
add Mamba when both attention masks are enabled (ablation)Ramba · RelBench zero-shot classificationAUROC (%): 72.8 -> 72.8No clear effect
add sparse attention masks (intra-entity and relational) on top of Mamba (ablation)Ramba · RelBench zero-shot classificationAUROC (%): 68.5 -> 72.8Improved
use schema-based gating instead of no gating (ablation)Ramba · RelBench zero-shot regressionR2 (%): +2.7% R2Improved
shuffle column names in the gate (ablation)Ramba gating · RelBench zero-shotAUROC/R2Hurt
remove column names entirely (ablation)Ramba · RelBench zero-shot classificationAUROC (%): 72.5 -> 70.9Hurt
use column-wise serialization instead of row-wise or random (ablation)Ramba · RelBenchretained performance: 100% vs row 88%, random 85%Improved
shuffle within columns (ablation)Ramba serialization · RelBenchretained performance: 95% retainedNo clear effect
use width-limited BFS with FK prioritization instead of DFS (ablation)Ramba sampling · RelBench regressionR2 (%): 16.8 -> 25.2Improved
use width-limited BFS with FK priority instead of full BFS (ablation)Ramba sampling · RelBench classificationAUROC (%): 70.3 -> 72.8Improved
increase context length from 512 to 4096 (ablation)Ramba · RelBench regressionR2 (%): 24.0 -> 26.0Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy