Research map / Architectures and efficiency / CNN design
Depthwise Separable Convolutions for Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace regular convolutions with depthwise separable convolutions (ablation) | ByteNet-like model · WMT EN-DE newstest2013 | Accuracy: 62.41 -> 63.87 | Helped |
| replace regular convolutions with depthwise separable convolutions (ablation) | ByteNet-like model · WMT EN-DE newstest2013 | Parameters (Non-Emb.): 314 M (230 M) -> 196 M (112 M) | Helped |
| reduce dilation and increase filter sizes to 3-7-7-7 (ablation) | Full-separability ByteNet-like model · WMT EN-DE newstest2013 | Accuracy: 63.87 -> 64.37 | Helped |
| use larger filters 3-7-15-31 and remove dilation (ablation) | Full-separability ByteNet-like model · WMT EN-DE newstest2013 | Accuracy: 63.87 -> 64.36 | Helped |
| replace full depthwise separability with grouped convolutions using 16 groups (ablation) | ByteNet-like model · WMT EN-DE newstest2013 | Accuracy: 63.87 -> 63.46 | Hurt |
| replace full depthwise separability with super-separable convolutions alternating 2 and 3 groups (ablation) | ByteNet-like model · WMT EN-DE newstest2013 | Accuracy: 64.36 -> 64.71 | Helped |
| use super-separable convolutions instead of full separability at higher feature depth (ablation) | SliceNet · WMT EN-DE newstest2013 | Accuracy: 66.71 -> 67.27 | Helped |
| use a large SliceNet with super-separable convolutions (ablation) | SliceNet · WMT EN-DE newstest2014 | BLEU: 25.5 -> 26.1 | Helped |
Closest papers by meaning
- Xception: Deep Learning With Depthwise Separable Convolutions (CVPR 2017 · similarity 0.86)
- Deep Transformers with Latent Depth (NeurIPS 2020 · similarity 0.86)
- Recurrent Stacking of Layers for Compact Neural Machine Translation Models (AAAI 2019 · similarity 0.86)
- Convolutional Sequence to Sequence Learning (ICML 2017 · similarity 0.85)
- Time-aware Large Kernel Convolutions (ICML 2020 · similarity 0.85)
- SliceGPT: Compress Large Language Models by Deleting Rows and Columns (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.