Research map / Label-efficient and robust learning / OOD and anomaly detection
MASKER: Masked Keyword Regularization for Reliable Text Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MASKER to BERT | BERT · 20 Newsgroups → SST-2 OOD | AUROC: 87.0 -> 98.6 | Helped |
| Add MASKER to BERT | BERT · Amazon → Fine Food OOD | AUROC: 78.5 -> 84.9 | Helped |
| Add MASKER to BERT | BERT · Fine Food → SST-2 | accuracy: 67.6 -> 80.0 | Helped |
| Add MASKER to BERT | BERT · Fine Food → IMDB | accuracy: 77.3 -> 85.6 | Helped |
| Use attention-selected keywords rather than random or frequency-selected keywords (ablation) | MASKER · Amazon 50 class reviews (25% ID classes) | AUROC: 80.52\stdv{0.72}, 81.32\stdv{0.78}, 83.33\stdv{0.44} -> 85.48\stdv{0.54} | Helped |
| Add MKR to attention-keyword MASKER (ablation) | MASKER with attention keywords · Amazon 50 class reviews (25% ID classes) | AUROC: 82.60\stdv{0.51} -> 85.48\stdv{0.54} | Helped |
| Add MER to attention-keyword MASKER (ablation) | MASKER with attention keywords · Amazon 50 class reviews (25% ID classes) | AUROC: 83.33\stdv{0.44} -> 85.48\stdv{0.54} | Helped |
| Replace MKR reconstruction with classification loss on keyword-masked documents (ablation) | MASKER · Not specified | performance | Hurt |
| Regularize attention weights directly to be uniform (ablation) | Vanilla models · Classification and OOD detection | performance | Hurt |
Closest papers by meaning
- Scaling up Masked Diffusion Models on Text (ICLR 2025 · similarity 0.85)
- Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More (ICML 2025 · similarity 0.84)
- SimReg: Achieving Higher Performance in the Pretraining via Embedding Similarity Regularization (NeurIPS 2026 · similarity 0.84)
- Data-Constrained Language Model Pretraining: Improved Regularization and Scaling Laws (NeurIPS 2026 · similarity 0.84)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.84)
- MaskLLM: Learnable Semi-Structured Sparsity for Large Language Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.