Research map / Architectures and efficiency / RNNs and normalization
Compressing Word Embeddings via Deep Compositional Code Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace independent GloVe word vectors with 16 × 32 compositional codes | Sentiment LSTM · IMDB test | accuracy: 87.18 -> 87.37 | Helped |
| Increase codebooks from 16 × 32 to 64 × 8 (ablation) | Compositional word embeddings · IMDB test | accuracy: 87.37 -> 88.15 | Helped |
| Use 8 × 64 coding instead of 16 × 32 coding (ablation) | Compositional word embeddings · IMDB test | accuracy: 87.37 -> 86.66 | Hurt |
| Replace the NMT baseline embeddings with 64 × 16 compositional codes | IWSLT14 De → En NMT · IWSLT14 De → En | BLEU: 29.45 -> 29.56 | Helped |
| Use 32 × 16 coding instead of 64 × 16 coding (ablation) | Compositional word embeddings · IWSLT14 De → En | BLEU: 29.56 -> 29.04 | Hurt |
| Prune 95% instead of 90% of embedding connections (ablation) | NMT baseline · IWSLT14 De → En | BLEU: 29.34 -> 28.84 | Hurt |
| Replace the NMT baseline embeddings with 32 × 16 compositional codes | ASPEC En → Ja NMT · ASPEC En → Ja | BLEU: 37.93 -> 38.10 | Helped |
| Use 64 × 16 coding instead of 32 × 16 coding (ablation) | Compositional word embeddings · ASPEC En → Ja | BLEU: 38.10 -> 38.89 | Helped |
| Prune 98% instead of 90% of embedding connections (ablation) | NMT baseline · ASPEC En → Ja | BLEU: 38.56 -> 37.09 | Hurt |
| Increase the number of codebooks M while holding K fixed (ablation) | Compositional code learning · GloVe embedding reconstruction validation set | reconstruction loss: 29.1 -> 15.5 (K=8); 27.0 -> 11.5 (K=16); 24.4 -> 9.3 (K=32); 21.9 -> 7.6 (K=64) | Helped |
| Increase codewords K while holding M fixed (ablation) | Compositional code learning · GloVe embedding reconstruction validation set | reconstruction loss: 29.1 -> 21.9 (M=8); 25.8 -> 16.9 (M=16); 21.9 -> 12.1 (M=32); 15.5 -> 7.6 (M=64) | Helped |
Closest papers by meaning
- Online Embedding Compression for Text Classification Using Low Rank Matrix Factorization (AAAI 2019 · similarity 0.89)
- Accelerating Neural Machine Translation with Partial Word Embedding Compression (AAAI 2021 · similarity 0.88)
- On the Downstream Performance of Compressed Word Embeddings (NeurIPS 2019 · similarity 0.87)
- LittleBit: Ultra Low-Bit Quantization via Latent Factorization (NeurIPS 2025 · similarity 0.84)
- Language Modeling Is Compression (ICLR 2024 · similarity 0.84)
- SECodec: Structural Entropy-based Compressive Speech Representation Codec for Speech Language Models (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.