Research map / Architectures and efficiency / Attention and transformers
AMOM: Adaptive Masking over Masking for Conditional Masked Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add AMOM adaptive masking to CMLM | CMLM · WMT16 EN→RO | BLEU: 33.46 -> 34.62 | Helped |
| Add AMOM adaptive masking to CMLM | CMLM · WMT16 RO→EN | BLEU: 33.83 -> 34.82 | Helped |
| Add AMOM adaptive masking to CMLM | CMLM · WMT14 EN→DE | BLEU: 27.21 -> 27.57 | Hurt |
| Add AMOM adaptive masking to CMLM | CMLM · WMT14 DE→EN | BLEU: 31.03 -> 31.67 | Helped |
| Add AMOM adaptive masking to CMLM | CMLM · XSUM | ROUGE-1: 25.80 -> 31.59 | Helped |
| Add AMOM adaptive masking to CMLM | CMLM · Py150 Python, 10 iterations | BLEU: 53.44 -> 56.50 | Helped |
| Add AMOM adaptive masking to CMLM | CMLM · GitHub-Java, 10 iterations | BLEU: 62.82 -> 65.43 | Helped |
| Add adaptive X masking (ablation) | CMLM · IWSLT14 DE→EN | BLEU: 33.87 -> 34.48 | Helped |
| Add MLM loss alongside adaptive X masking (ablation) | CMLM · IWSLT14 DE→EN | BLEU: 34.48 -> 34.57 | Helped |
| Replace adaptive X masking with JM-NAT masking (ablation) | CMLM · IWSLT14 DE→EN | BLEU: 34.48 -> 34.13 | Hurt |
| Replace adaptive X masking with mix cutoff (ablation) | CMLM · IWSLT14 DE→EN | BLEU: 34.48 -> 33.96 | Hurt |
| Use adaptive Y masking rather than uniform second-step masking (ablation) | CMLM with adaptive X masking · IWSLT14 DE→EN | BLEU: 34.53 -> 34.84 | Helped |
Closest papers by meaning
- Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More (ICML 2025 · similarity 0.88)
- Re-evaluating Confidence Remasking in Masked Diffusion Language Models (NeurIPS 2026 · similarity 0.87)
- MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models (NeurIPS 2024 · similarity 0.87)
- Enhancing Large Language Model Performance with Gradient-Based Parameter Selection (AAAI 2025 · similarity 0.87)
- Simple and Effective Masked Diffusion Language Models (NeurIPS 2024 · similarity 0.87)
- Fast and Fluent Diffusion Language Models via Convolutional Decoding and Rejective Fine-tuning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.