Research map / Architectures and efficiency / Quantization

Structured Transforms for Low-Overhead Quantization of Language Models

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Revisits Kashin-decomposition weight quantization for LLMs with a partitioned greedy algorithm, sign-randomized DCT transform, and closed-form k-means initialization, reporting near-OPTQ/QuIP accuracy at 4-bit and stability on stress models.

Problem

Prior Kashin-decomposition LLM quantization suffered from convergence failures under matrix reformulation, non-separating four-peak distributions, O(N^2) random orthogonal transforms, and slow multi-restart k-means clustering.

Approach

A greedy algorithm with alternating updates in blocks of four (two steps from the cube vertices, two from the rotated dictionary) that guarantees four-peak factor distributions, uses a sign-randomized DCT as the orthogonal transform, and initializes k-means at analytic centroids ±c1±c2.

Key results

• At 4-bit per channel on Pythia-6.9B, QuIP and QuIP-RG diverge to >2000 Wiki-2 PPL and QuIP# reaches ~325, while Kashin-DCT+H reaches 20.64 vs OPTQ 12.02 and FP16 11.41.
• On Mistral-7B v0.1, GPTQ degrades to ~380 Wiki-2 PPL and QuIP variants abort with NaN, while Kashin-DCT+H gets 8.92 vs FP16 8.63.
• On Llama-2-7B, Kashin-DCT+H attains 9.60 Wiki-2 PPL vs OPTQ 9.80 and QuIP# 9.47 at 4 bits; quantization time for Llama-2-7B is 1313.6 s vs OPTQ 229.9 s.

Surprising findings

• Kashin-DCT+H is slower than OPTQ and QuIP# to quantize on Llama-2-7B (1313.6 s vs 229.9 s and 763.0 s), despite the claimed favorable scaling.
• QuIP variants with Kronecker incoherence are unstable on several models, while Kashin-DCT remains numerically stable on Mistral-7B where all QuIP variants abort with NaN.

Limitations

• Convergence bound for the new algorithm is weaker than the original by a polylogarithmic/beta factor; theory is vector-level only.
• Quantization wall-clock time for Kashin-DCT+H is higher than OPTQ and QuIP# on Llama-2-7B.
• Does not compare with activation-aware methods (AWQ, OmniQuant) or rotation-based methods, and evaluated only at 4-bit per channel.

Experimental setup

Datasets: WikiText-2 (calibration, 1000 sequences of length 2048)

Benchmarks: WikiText-2 C4 HellaSwag PiQA Winogrande

Metrics: perplexity accuracy acc_norm wall-clock time

Baselines: RTN OPTQ QuIP QuIP-RG QuIP#

Builds on: Kashin decomposition OPTQ QuIP QuIP# Kashin greedy algorithm (Lyubarskii-Vershynin style)

Closest papers by meaning

• Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.91)
• OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting (ICLR 2025 · similarity 0.90)
• Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models (ICLR 2024 · similarity 0.89)
• ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization (AAAI 2025 · similarity 0.89)
• Treasures in Discarded Weights for LLM Quantization (AAAI 2025 · similarity 0.89)
• AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024 · similarity 0.89)