AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

PinTok: Tokenizers Deserve Dedicated Pinned CPU-Compute and Memory

Sean Choi, Myungheon Chin, Ernest K. Ryu

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Efficient LLM serving

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PinTok, a system that runs LLM tokenizers on dedicated pinned CPU compute and memory to cut inference overhead from scheduling, core selection, and data copying.

Problem

Tokenization in LLM inference runs as a standard CPU process, incurring overhead from scheduling delays, core selection, data copying, and other system-level costs.

Approach

Treats the tokenizer as a component worth accelerating by giving it dedicated pinned CPU cores and memory rather than running it as a generic OS process.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use PinTok pinned tokenizer instead of Python tokenizertokenizer system · ChromaDB retrievalaverage latency (ms): 20.06 -> 11.35Improved
use PinTok instead of Python tokenizertokenizer system · ChromaDB insertionaverage latency (ms): 56.26 -> 47.55Improved
use PinTok instead of Python tokenizertokenizer system · Qwen3 4B query (TTFT)average latency (ms): 422.26 -> 413.55Improved
use PinTok instead of Python tokenizertokenizer system · GPT-OSS 120b query (TTFT)average latency (ms): 1357.26 -> 1348.55No clear effect
use PinTok instead of Rust tokenizertokenizer system · ChromaDB retrievalaverage latency (ms): 16.73 -> 11.35Improved
use PinTok in tokenizer benchmarktokenizer system · Tokenizer benchmarklatency and throughput: up to 95% avg latency reduction, 2,084% throughputImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy