Research map / Architectures and efficiency / Quantization
Embedding Compression with Isotropic Iterative Quantization
Siyu Liao, Jie Chen, Yanzhi Wang, Qinru Qiu, Bo Yuan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use isotropy-oriented IIQ instead of ITQ at compression ratio 32 | GloVe embeddings · Word similarity benchmarks | Spearman correlation | Improved |
| Use isotropy-oriented IIQ instead of ITQ at compression ratio 32 | GloVe embeddings · Categorization benchmarks | purity | Improved |
| Use isotropy-oriented IIQ instead of ITQ at compression ratio 32 | GloVe embeddings · Topic classification benchmarks | accuracy | Improved |
| Increase compression ratio from 32 to 64 (ablation) | IIQ embeddings · Word similarity benchmarks | Spearman correlation: IIQ-32 to IIQ-64 values shown in table | Hurt |
| Increase compression ratio from 32 to 128 (ablation) | IIQ embeddings · Word similarity benchmarks | Spearman correlation: IIQ-32 to IIQ-128 values shown in table | Hurt |
| Increase compression ratio from 32 to 64 or 128 (ablation) | IIQ embeddings · Topic classification | accuracy: IIQ-32, IIQ-64, and IIQ-128 values shown in table | Hurt |
| Increase compression ratio from 32 to 64 (ablation) | IIQ embeddings · TR9856 | Spearman correlation: 9.80 to 12.81 | Improved |
| Increase compression ratio from 32 to 128 (ablation) | IIQ embeddings · TR9856 | Spearman correlation: 9.80 to 12.31 | Improved |
| Compress GloVe with IIQ-32 rather than keep original embeddings | GloVe · MEN | Spearman correlation: 73.62 -> 76.43 | Improved |
Closest papers by meaning
- Online Embedding Compression for Text Classification Using Low Rank Matrix Factorization (AAAI 2019 · similarity 0.88)
- Compressing Word Embeddings via Deep Compositional Code Learning (ICLR 2018 · similarity 0.87)
- On the Downstream Performance of Compressed Word Embeddings (NeurIPS 2019 · similarity 0.87)
- Variational Bayesian Quantization (ICML 2020 · similarity 0.87)
- Extreme Compression of Large Language Models via Additive Quantization (ICML 2024 · similarity 0.85)
- CASP: Compression of Large Multimodal Models Based on Attention Sparsity (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
