AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

TileQ: Efficient Low-Rank Quantization of Mixture-of-Experts with 2D Tiling

Hongyaoxing Gu, Xinzhe Chen, Lijuan Hu, Liu fangfang

ICML 2026 · Architectures and efficiency › Mixture of experts · approach: MoE LLM compression · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add low-rank component (ablation)TileQ-v (Qwen1.5-MoE-A2.7B) · WikiText-2 2-bitperplexity: 12.5 -> 7.35Improved
add low-rank component (ablation)TileQ-v (Qwen1.5-MoE-A2.7B) · WikiText-2 3-bitperplexity: 7.98 -> 6.93Improved
use vector quantization instead of scalar (ablation)TileQ (Qwen1.5-MoE-A2.7B) · WikiText-2 2-bitperplexity: 7.56 -> 7.35Improved
use vector quantization instead of scalar (ablation)TileQ (Qwen1.5-MoE-A2.7B) · WikiText-2 3-bitperplexity: 6.94 -> 6.93No clear effect
add rotation (ablation)TileQ-v (Qwen1.5-MoE-A2.7B) · WikiText-2 2-bitperplexity: 7.6 -> 7.35Improved
use 2D-tiling instead of per-expert low-rank (ablation)TileQ-v (Qwen1.5-MoE-A2.7B) · WikiText-2 2-bitperplexity: 7.49 -> 7.35No clear effect
use 2D-tiling instead of per-expert low-rank (ablation)TileQ-v · extra bits per weightextra bit-width: 0.31 -> 0.04Improved
use TileQ-v instead of GPTQMixtral-8x7B · WikiText-2 2-bitperplexity: 15.3 -> 4.78Improved
use TileQ-s instead of LoPRoMixtral-8x7B · WikiText-2 3-bitperplexity: 4.13 -> 4.10No clear effect
use vector instead of scalar TileQ at 3-bitTileQ · 3-bit quantizationaccuracyNo clear effect
use fused LoTileMoE 2D-tiling low-rank inference instead of element-wise or 1D MiLoTileQ · MoE MLP block latency prefilllatency: less than 5%Improved
use 1D-shared (MiLo) or element-wise low-rank at decodeMiLo-1D, element-wise · Qwen3-MoE decodelatency: around 75% latencyHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy