Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Bayesian Attention Belief Networks
Shujian Zhang, Xinjie Fan, Bo Chen, Mingyuan Zhou
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace deterministic attention with BABN | ALBERT-base · GLUE and SQuAD in-domain tasks | accuracy | Improved |
| Replace deterministic attention with BABN | BERT-base and RoBERTa-base · NLI, paraphrase detection, and commonsense reasoning; ID and OD | accuracy | Improved |
| Replace deterministic attention with BABN | BERT-base and RoBERTa-base · NLI, paraphrase detection, and commonsense reasoning; ID and OD | ECE | Improved |
| Replace deterministic attention with BABN | ALBERT-base · Textfooler, Textbugger, and BAE attacks | failed attack percentage | Mixed |
| Replace deterministic attention with BABN | IWSLT LSTM translation model · IWSLT | BLEU: 32.77 -> 34.23 | Improved |
| Replace deterministic attention with BABN | MCAN · VQA-v2 Original and Noisy | accuracy: 66.74 -> 66.92; 63.58 -> 64.40 | Improved |
| Replace deterministic attention with BABN | MCAN · VQA-v2 Original and Noisy | PAvPU: 71.96 -> 72.21; 68.29 -> 70.43 | Improved |
| Add the upward-path contribution by using ρ=1.5 instead of ρ=0 (ablation) | BABN on MCAN · VQA-v2 Original | accuracy: 44.62 -> 66.92 | Improved |
| Add the σ-weighted contribution by using σ=1.00e^{-6} instead of σ=0 (ablation) | BABN on MCAN · VQA-v2 Original | accuracy: 66.78 -> 66.92 | Improved |
| Use the deterministic-upward and stochastic-downward inference structure (ablation) | BABN · VQA | accuracy | Improved |
Closest papers by meaning
- Bayes optimal learning of attention-indexed models (NeurIPS 2025 · similarity 0.85)
- Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration (ICML 2024 · similarity 0.85)
- MVA: Linear Attention with High-order Query-Keys Integration and Multi-level Vocabulary Decomposition (ICML 2025 · similarity 0.85)
- SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking (NeurIPS 2026 · similarity 0.85)
- Attention-aware Inference Optimizations for Large Vision-Language Models with Memory-efficient Decoding (CVPR 2026 · similarity 0.85)
- Fundamental limits of learning in sequence multi-index models and deep attention networks: high-dimensional asymptotics and sharp thresholds (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
