AI Paper Atlas

Research map / Architectures and efficiency / Quantization

Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks

Minyoung Huh, Brian Cheung, Pulkit Agrawal, Phillip Isola

ICML 2023 · Architectures and efficiency › Quantization · approach: Post-training weight quantization · 9 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add the full combination of affine re-parameterization, synchronized updates, alternating optimization, and replacementVQ AlexNet, ResNet18, and ViT(T) · ImageNet100Accuracy: 47.2 -> 57.9; 64.1 -> 71.0; 48.6 -> 56.7Improved
Add affine re-parameterization (ablation)VQ AlexNet, ResNet18, and ViT(T) · ImageNet100Accuracy: 47.2 -> 55.7; 64.1 -> 70.4; 48.6 -> 52.8Improved
Add synchronized commitment updates (ablation)VQ AlexNet, ResNet18, and ViT(T) · ImageNet100Accuracy: 47.2 -> 49.2; 64.1 -> 66.0; 48.6 -> 51.3Improved
Add alternating optimization (ablation)VQ AlexNet, ResNet18, and ViT(T) · ImageNet100Accuracy: 47.2 -> 52.8; 64.1 -> 67.5; 48.6 -> 53.1Improved
Add l2 normalizationVQ classification models · ImageNet100AccuracyHurt
Add affine re-parameterization, alternating optimization, and replacementVQVAE · CIFAR10MSE: 5.65 -> 1.81Improved
Also add l2 normalization to the proposed VQVAE combination (ablation)VQVAE + Affine + OPT + Replace · CIFAR10MSE: 1.81 -> 1.74Improved
Add affine re-parameterization, alternating optimization, and replacementVQVAE · CelebAMSE: 10.02 -> 3.84Improved
Also add l2 normalization to the proposed VQVAE combination (ablation)VQVAE + Affine + OPT + Replace · CelebAMSE: 3.84 -> 4.42Hurt
Add affine re-parameterization and alternating optimizationMaskGIT · CelebAFID: 90.4 -> 74.8Improved
Increase alternating optimization inner steps from 1 to 8 (ablation)AlexNet · ImageNet100Accuracy: 52.54 (+5.58) -> 58.03 (+11.09)Improved
Increase alternating optimization outer steps (ablation)AlexNet · ImageNet100Accuracy: 55.13 (+8.19) -> 55.09 (+8.15); 57.45 (+10.51) -> 55.29 (+8.35)No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy