Research map / Architectures and efficiency / Attention and transformers
NADIR: Differential Attention Flow for Non-Autoregressive Transliteration in Indic Languages
Lakshya Tomar, Vinayak Abrol, Puneet Agarwal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard attention with differential attention (ablation) | NAR transliteration encoder · Aksharantar Roman->Indic mean | CER: 21.88 -> 16.12 | Improved |
| add MoE to differential-attention NAR (ablation) | Diff NAR · Aksharantar Roman->Indic mean | CER: 16.12 -> 15.78 | Improved |
| add MoE to differential-attention NAR (ablation) | Diff NAR · Aksharantar Roman->Indic mean | word accuracy: 46.89 -> 50.13 | Improved |
| use differential attention instead of standard (ablation) | NAR transliteration · Hallucination counts | substitution errors: 72,127 -> 57,349 | Improved |
| use differential attention instead of standard (ablation) | NAR transliteration · Hallucination counts | insertion errors: 28,454 -> 27,682 | No clear effect |
| add MoE to Diff NAR (ablation) | Diff NAR · Hallucination counts | omission errors: 23,453 -> 25,334 | Hurt |
| add MoE to Diff NAR (ablation) | Diff NAR · Hallucination counts | insertion errors: 27,682 -> 23,654 | Improved |
| add MoE to Diff NAR (ablation) | Diff NAR · Hallucination counts | repetition errors: 4,126 -> 3,186 | Improved |
| use NADIR (NAR) instead of AR IndicXLIT | Roman->Indic transliteration · Aksharantar Roman->Indic mean | inference time (s): 116.48 -> 8.95 | Improved |
| use NADIR (NAR) instead of AR IndicXLIT | Roman->Indic transliteration · Aksharantar Roman->Indic mean | CER: 14.44 -> 15.78 | Hurt |
Closest papers by meaning
- Improving Non-Autoregressive Translation Models Without Distillation (ICLR 2022 · similarity 0.87)
- When Linear Attention Meets Autoregressive Decoding: Towards More Effective and Efficient Linearized Large Language Models (ICML 2024 · similarity 0.84)
- Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models (ICLR 2025 · similarity 0.84)
- Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flows (NeurIPS 2025 · similarity 0.83)
- AEQA-NAT : Adaptive End-to-end Quantization Alignment Training Framework for Non-autoregressive Machine Translation (ICML 2025 · similarity 0.83)
- LAMA-UT: Language Agnostic Multilingual ASR Through Orthography Unification and Language-Specific Transliteration (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
