Research map / Architectures and efficiency / RNNs and normalization
Implicit Feature Decoupling With Depthwise Quantization
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Decompose feature tensors into depthwise sub-tensors and quantize each with its own codebook rather than use coupled VQ | DQ-AE · CIFAR-10 | NLL (bits/dim): 4.36 -> 3.14 bits/dim | Helped |
| Decompose feature tensors along the channel axis rather than the spatial axis (ablation) | DQ on VGG-16 features · ImageNet VGG-16 features, K=32 | L_DQ: 0.192±0.002 -> 0.184±0.001 | Helped |
| Decompose feature tensors along the channel axis rather than the spatial axis (ablation) | DQ on VGG-16 features · ImageNet VGG-16 features, K=1024 | mean entropy: 3.64±0.01 -> 3.99±0.01 | Helped |
| Use multiple codebooks instead of one (increase M) (ablation) | DQ-AE · CIFAR-10 | cross-entropy (nats/dim): At K=512: M=1 3.85e+00 -> M=10 2.13e+00 | Helped |
| Use multiple codebooks instead of one (increase M) (ablation) | DQ-AE · CIFAR-10 | L2 reconstruction loss: At K=512: M=1 6.78e-02 -> M=10 1.68e-03 | Helped |
| Use one codebook (M=1), making DQ equivalent to VQ (ablation) | DQ-AE · CIFAR-10 | cross-entropy (nats/dim): 4.16e+00 vs 4.16e+00; 4.01e+00 vs 4.01e+00; 3.85e+00 vs 3.85e+00 | No clear effect |
| Increase codebook size K while keeping M fixed (ablation) | DQ-AE · CIFAR-10 | cross-entropy (nats/dim): M=10: 2.71e+00 (K=32) -> 2.13e+00 (K=512) | Helped |
| Use lower-capacity codebooks in a hierarchical DQ-AE (ablation) | Hierarchical DQ-AE · CIFAR-10 | cross-entropy (nats/dim): 2.84e+00 for [256,128]; 3.21e+00 for [32,128] | Mixed |
| Use depthwise quantization instead of VQ in the hierarchical model | Hierarchical DQ-AE · CIFAR-10, M=5, cross-entropy | nats/dim: For [256,128]: 3.63 -> 2.84 | Helped |
| Use depthwise quantization instead of VQ in the hierarchical model | Hierarchical DQ-AE · CIFAR-10, M=5, L2 reconstruction | L2 reconstruction loss: For [256,128]: 4.52e-02 -> 9.31e-03 | Helped |
| Use depthwise quantization instead of VQ on MNIST | DQ · MNIST | L2 reconstruction loss: 3.41e-04 -> 1.92e-04 | Helped |
| Use depthwise quantization instead of VQ on CelebA | DQ · CelebA | L2 reconstruction loss: 3.70e-02 -> 9.57e-03 | Helped |
Closest papers by meaning
- Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks (ICML 2020 · similarity 0.88)
- Deep Neural Network Quantization via Layer-Wise Optimization Using Limited Training Data (AAAI 2019 · similarity 0.88)
- DQT: Dynamic Quantization Training via Dequantization-Free Nested Integer Arithmetic (AAAI 2026 · similarity 0.88)
- Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks (CVPR 2018 · similarity 0.88)
- Quantization without Tears (CVPR 2025 · similarity 0.87)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.