Research map / Architectures and efficiency / RNNs and normalization
Synchronizing Probabilities in Model-Driven Lossless Compression
Aviv Adler, Jennifer Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PMATIC with delta=1e-5 instead of plain arithmetic coding | LLaMA 3.1 8B compression · Enwik8 | compression ratio: 0.0780 -> 0.0847 | Hurt |
| increase robustness setting to delta=1e-2, r=0.125 (ablation) | PMATIC with LLaMA 3.1 8B · Enwik8 | compression ratio: 0.0847 -> 0.2492 | Hurt |
| use PMATIC at most robust setting instead of gzip level 9 | LLaMA 3.1 8B compression · Enwik8 | compression ratio: 0.4601 -> 0.2492 | Improved |
| use PMATIC at most robust setting instead of cmix | LLaMA 3.1 8B compression · Enwik8 | compression ratio: 0.3558 -> 0.2492 | Improved |
| use PMATIC delta=1e-2 for Qwen2.5 on Chinese text vs cmix | Qwen2.5 7B Instruct · Dream of the Red Chamber | compression ratio: 0.3824 -> 0.3073 | Improved |
| use PMATIC delta=1e-3 for decoding across two different laptops | LLaMA 3.1 8B · Cross-machine decoding of 100 Wikipedia articles and Emma excerpts | files decoded correctly: 0 files decoded | Hurt |
| use PMATIC delta=1e-2 for decoding across two different laptops | LLaMA 3.1 8B · Cross-machine decoding | files decoded correctly: all files decoded | Improved |
| add synthetic logit noise within [-2 delta, 2 delta] | PMATIC · Synthetic non-determinism | decoding success: all files decoded | No clear effect |
Closest papers by meaning
- Efficient Mismatch-Tolerant Coding for Model-Driven Compression (ICML 2026 · similarity 0.95)
- Lossless Compression with Probabilistic Circuits (ICLR 2022 · similarity 0.85)
- Scalable Model Compression by Entropy Penalized Reparameterization (ICLR 2020 · similarity 0.85)
- "Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approach (NeurIPS 2022 · similarity 0.85)
- Language Models as Zero-shot Lossless Gradient Compressors: Towards General Neural Parameter Prior Models (NeurIPS 2024 · similarity 0.84)
- Benchmarking Predictive Coding Networks -- Made Simple (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
