AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

TeZO: Empowering the Low-Rankness on the Temporal Dimension in the Zeroth-Order Optimization for Fine-Tuning LLMs

Yan Sun, Qixin Zhang, Liang Ding, Shen Li, Dacheng Tao

TPAMI 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Adapter-based parameter-efficient tuning · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Adam-style adaptivity with low-rank temporal structure (TeZO-Adam vs TeZO) (ablation)TeZO · OPT-13Baverage gap vs FT: -5.2 -> -2.5Improved
add momentum (TeZO-m vs TeZO) (ablation)TeZO · OPT-13Baverage gap vs FT: -5.2 -> -4.8Improved
use TeZO-Adam instead of MeZO-AdamTeZO-Adam · OPT-13Baverage gap vs FT: -3.2 -> -2.5Improved
use TeZO-Adam instead of MeZO-AdamTeZO-Adam · LLaMA-7Baverage gap vs FT: -7.5 -> -7.0Improved
use TeZO instead of MeZOTeZO · LLaMA-7Baverage gap vs FT: -9.2 -> -9.2No clear effect
use TeZO instead of MeZOTeZO · RoBERTa-large k=16average gap vs FT: -8.7 -> -9.3Hurt
use TeZO instead of MeZOTeZO · RoBERTa-large k=512average gap vs FT: -3.3 -> -3.6Hurt
use TeZO instead of MeZOTeZO · OPT-13Baverage gap vs FT: -5.4 -> -5.2Improved
use TeZO-Adam instead of MeZO-AdamTeZO-Adam · OPT-13B RTE memoryGPU memory: 75.27 G -> 26.01 GImproved
use TeZO-Adam instead of MeZO-AdamTeZO-Adam · OPT-13B RTE wall-clocktime per iteration: 642 ms -> 394 msImproved
use TeZO instead of MeZO on small modelsTeZO · OPT-125M wall-clocktime per iteration: 33 ms -> 36 msHurt
use TeZO instead of MeZO on large modelsTeZO · OPT-30B wall-clocktime per iteration: 871 ms -> 680 msImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy