Research map / Architectures and efficiency / Quantization
Hyper-Compression: Model Compression via Hyperfunction
Fenglei Fan, Juntong Fan, Dayang Wang, Jingbo Zhang, Zelin Dong, Shijun Zhang, Ge Wang, Tieyong Zeng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Hyper-Compression to LLaMA2-7B (2.60x) | LLaMA2-7B · 8 downstream tasks average | accuracy (%): 65.88 -> 64.89 | Hurt |
| apply Hyper-Compression on top of pruned Sheared-LLaMA-1.3B | Sheared-LLaMA-1.3B · 8 downstream tasks average | accuracy (%): 50.31 -> 49.76 | Hurt |
| apply Hyper-Compression to Qwen2-72B-Instruct | Qwen2-72B-Instruct · 8 downstream tasks average | accuracy (%): 78.27 -> 78.31 | No clear effect |
| apply Hyper-Compression to LLaMA2-7B | LLaMA2-7B · WikiText-2 | perplexity: 5.47 -> 5.82 | Hurt |
| use Hyper-Compression instead of int4 asymmetric quantization | LLaMA2-7B · WikiText-2 | perplexity: 26160.34 -> 5.82 | Improved |
| use GPTQ int4 instead of Hyper-Compression | LLaMA2-7B · WikiText-2 | perplexity: 5.82 -> 5.69 | Improved |
| apply Hyper-Compression to LiteLLaMA | LiteLLaMA · WikiText-2 | perplexity: 31.82 -> 37.85 | Hurt |
| apply Hyper-Compression to UNet (7.87x) | UNet · UNet segmentation | Dice (%): 99.86 -> 99.71 | No clear effect |
| use Hyper-Compression instead of pruning on UNet | UNet · UNet segmentation | Dice (%): 96.34 -> 99.71 | Improved |
| use Hyper-Compression instead of pruning on MobileNetV3 | MobileNetV3 · MobileNetV3 classification | top-1 accuracy (%): 69.32 -> 73.93 | Improved |
| use Hyper-Compression instead of SQuant-INT4 on ResNet50 | ResNet50 · ResNet50 classification | top-1 accuracy (%): 70.80 -> 76.40 | Improved |
| use Hyper-Compression on Qwen1.5-7B-Chat (2.67x) | Qwen1.5-7B-Chat · 16 tasks average | accuracy (%): 65.23 -> 64.75 | Hurt |
Closest papers by meaning
- Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.87)
- In defense of parameter sharing for model-compression (ICLR 2024 · similarity 0.87)
- Unified Scaling Laws for Compressed Representations (NeurIPS 2025 · similarity 0.86)
- EvoPress: Accurate Dynamic Model Compression via Evolutionary Search (ICML 2025 · similarity 0.86)
- SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models (NeurIPS 2026 · similarity 0.86)
- Genomics Data Lossless Compression with (S, K)-Mer Encoding and Deep Neural Networks (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
