Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Bayesian Attention Modules
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace deterministic attention with BAM-WC | GAT · Cora, Citeseer, and PubMed | accuracy: 83.00 -> 83.81; 72.50 -> 73.52; 77.26 -> 78.82 | Helped |
| Use stochastic Weibull attention with contextual prior | MCAN · VQA-v2 validation, noisy | accuracy: 61.25 -> 62.89 | Helped |
| Use stochastic attention for uncertainty estimation | MCAN · VQA-v2 validation, original and noisy | PAvPU: 70.04 -> 71.21; 65.34 -> 66.75 | Helped |
| Replace deterministic soft attention with BAM-WC | Att2in · MS-COCO | BLEU-4: 32.2 -> 32.8 | Helped |
| Replace deterministic attention with BAM-WC | LSTM translation model · IWSLT | BLEU: 32.77 -> 33.81 | Helped |
| Use BAM-WC at finetuning | ALBERT-base · GLUE QQP | score: 90.8 -> 90.7 | Hurt |
| Use BAM-WC at finetuning | ALBERT-base · GLUE benchmarks | score: MRPC: 86.5 -> 88.5; CoLA: 54.5 -> 55.8; RTE: 75.8 -> 76.2; MNLI: 85.1 -> 85.6; QNLI: 90.9 -> 91.5; QQP: 90.8 -> 90.7; SS | Mixed |
| Add KL regularization to stochastic attention (ablation) | Bayesian attention on graph node classification · Cora, Citeseer, and PubMed | accuracy: 83.39 -> 83.81; 72.91 -> 73.52; 78.50 -> 78.82 | Helped |
| Use a contextual rather than fixed prior (ablation) | BAM with Weibull attention · Cora, Citeseer, and PubMed | accuracy: 83.48 -> 83.81; 73.18 -> 73.52; 78.50 -> 78.82 | Helped |
| Use Weibull rather than Lognormal attention (ablation) | BAM with contextual prior · Cora, Citeseer, and PubMed | accuracy: 83.34 -> 83.81; 73.04 -> 73.52; 78.76 -> 78.82 | Helped |
| Use Weibull rather than Lognormal attention (ablation) | BAM with contextual prior · MS-COCO | BLEU-4: 32.7 -> 32.8 | Helped |
| Use Weibull rather than Lognormal attention (ablation) | BAM with contextual prior · MS-COCO | CIDEr: 105.0 -> 104.5 | Hurt |
Closest papers by meaning
- Bayes optimal learning of attention-indexed models (NeurIPS 2025 · similarity 0.87)
- Fundamental limits of learning in sequence multi-index models and deep attention networks: high-dimensional asymptotics and sharp thresholds (ICML 2025 · similarity 0.85)
- SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking (NeurIPS 2026 · similarity 0.85)
- EAN: An Efficient Attention Module Guided by Normalization for Deep Neural Networks (AAAI 2024 · similarity 0.85)
- Sparse Attention as Compact Kernel Regression (NeurIPS 2026 · similarity 0.85)
- Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.