AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Preconditioned DeltaNet: Curvature-aware Sequence Modeling for Linear Recurrences

Neehal Tumma, Noel Loo, Daniela Rus

ICML 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Theory of neural network sensitivity · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add diagonal preconditioner (PGDN) to Gated DeltaNetGDN 340M · commonsense reasoning averageaccuracy: 48.86 -> 49.96Improved
add diagonal preconditioner (PKDA) to Kimi Delta AttentionKDA 340M · commonsense reasoning averageaccuracy: 49.12 -> 49.89Improved
add diagonal preconditioner (PDN) to DeltaNetDeltaNet 340M · commonsense reasoning averageaccuracy: 47.41 -> 47.96Improved
add diagonal preconditioner to DeltaNetDeltaNet 340M · in-context retrieval averageaccuracy: 22.59 -> 23.84Improved
add diagonal preconditioner to GDNGDN 340M · in-context retrieval averageaccuracy: 24.77 -> 26.17Improved
add diagonal preconditioner to KDAKDA 340M · in-context retrieval averageaccuracy: 25.88 -> 26.76Improved
add diagonal preconditioner to GDN at 1B scaleGDN 1B · commonsense reasoning and in-context retrieval averagesaccuracy: 55.44 -> 56.38 (CR); 33.87 -> 34.99 (ICR)Improved
add diagonal preconditioner to KDA at 1B scaleKDA 1B · commonsense reasoning and in-context retrieval averagesaccuracy: 55.43 -> 56.64 (CR); 34.55 -> 34.88 (ICR)Improved
add preconditioner to DeltaNet and GDNDN, GDN 340M · MQAR synthetic recallaccuracyImproved
add preconditioner to DeltaNet, GDN, KDA340M models · S-NIAH-1/2/3 (RULER)accuracyImproved
apply preconditioner to the write key (ATK on GDN) instead of the query (ATQ on Mamba-2) (ablation)preconditioned recurrence · training loss, 340Mtrain loss: 2.543 -> 2.536Improved
use stable squashed preconditioner parameterization Bt instead of raw inverse of At (ablation)PGDN · training loss, 340Mtrain loss: 2.536 -> 2.511Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy