Research map / Probabilistic and scientific modeling / Variational and generative sampling
Latent Alignment and Variational Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace soft attention with exact marginalization over categorical alignments | NMT and VQA models · IWSLT 2014 German-English | BLEU: 32.77 -> 33.29 | Helped |
| Replace soft attention with exact marginalization over categorical alignments | NMT and VQA models · VQA 2.0 | NLL: 1.76 -> 1.69 | Helped |
| Train hard categorical alignments using the Jensen lower bound | Soft Attention · IWSLT 2014 German-English | BLEU: 32.77 -> 31.40 | Hurt |
| Use categorical variational attention with enumeration | Soft Attention · IWSLT 2014 German-English | BLEU: 32.77 -> 33.68 | Helped |
| Use categorical variational attention with single-sample training | Soft Attention · IWSLT 2014 German-English | BLEU: 32.77 -> 33.30 | Helped |
| Use categorical variational attention | Soft Attention · VQA 2.0 | NLL: 1.76 -> 1.69 | Helped |
| Use categorical variational attention for task-specific VQA evaluation | Soft Attention · VQA 2.0 | Eval: 58.93 -> 58.44 | Hurt |
| Use categorical variational attention | Soft Attention · WMT 2017 English-German | BLEU: 24.10 -> 24.98 | Helped |
| Use Dirichlet relaxed alignments instead of categorical alignments (ablation) | Variational Attention · IWSLT 2014 German-English | BLEU: 33.30 -> 30.05 | Hurt |
| Use K-Max decoding with K=5 instead of exact enumeration (ablation) | Variational Attention · IWSLT 2014 German-English | PPL: 6.08 -> 6.42 | Hurt |
| Replace REINFORCE with RWS using 5 samples (ablation) | Variational Attention · IWSLT 2014 German-English | PPL: 6.17 -> 6.41 | Hurt |
| Replace REINFORCE with Gumbel-Softmax (ablation) | Variational Attention · IWSLT 2014 German-English | PPL: 6.17 -> 6.51 | Hurt |
Closest papers by meaning
- Mind the Heads: Topological Representation Alignment for Multimodal LLMs (NeurIPS 2026 · similarity 0.86)
- Towards Multimodal Large Language Models with Both Training and Inference Efficiency (ICML 2026 · similarity 0.86)
- Low-Rank Approximation for Sparse Attention in Multi-Modal LLMs (CVPR 2024 · similarity 0.86)
- Scale-invariant attention (NeurIPS 2025 · similarity 0.86)
- Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.86)
- Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.