AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

A Modern Self-Referential Weight Matrix That Learns to Modify Itself

Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber

ICML 2022 · Architectures and efficiency › RNNs and normalization · approach: Neural-symbolic reasoning · 15 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use self-modifying SRWM instead of DeltaNetfew-shot classifier · Mini-ImageNet 5-way 5-shotaccuracy: 62.7 ± 0.1 -> 61.4 ± 0.1Hurt
use self-modifying SRWM instead of DeltaNetsequential adaptation model · Mini-ImageNet after Omniglottotal accuracy: 50.4 -> 53.3Improved
use self-modifying SRWM instead of DeltaNetProcGen agent · 6 environments easy, testnormalized aggregated score: 18.6 (1.7) -> 20.0 (1.8)Improved
remove self-modification (ablation)SRWM ProcGen agent · 6 environments easy, testnormalized aggregated score: 20.0 (1.8) -> 15.3 (1.9)Hurt
add persistent weight updates across the episode instead of resetting every backpropagation span (ablation)SRWM ProcGen agent · 6 environments easy, testnormalized aggregated score: 16.1 (2.2) -> 20.0 (1.8)Improved
replace DeltaNet slow weights with SRWMDeltaNet · ProcGen 4 environments memory, trainnormalized aggregated score: 51.8 (2.6) -> 59.0 (2.1)Improved
replace DeltaNet slow weights with SRWMDeltaNet · ProcGen 4 environments memory, testnormalized aggregated score: 38.0 (4.1) -> 38.5 (3.2)No clear effect
use SRWM instead of DeltaNetfew-shot classifier · Omniglot 5-way 1-shotaccuracy: 97.2 ± 0.0 -> 97.4 ± 0.0No clear effect
use SRWM instead of DeltaNetfew-shot classifier · Mini-ImageNet 5-way 1-shotaccuracy: 47.0 ± 0.1 -> 47.0 ± 0.2No clear effect
use SRWM instead of DeltaNetfew-shot classifier · FC100 5-way 5-shotaccuracy: 43.2 ± 0.1 -> 44.5 ± 0.1No clear effect
increase vision backend from Conv-4-32 to Res-12-256 (ablation)SRWM · FC100 5-way 5-shotaccuracy: 44.5 ± 0.1 -> 46.3 ± 0.1Improved
use Res-12-512 instead of Res-12-256 (ablation)SRWM · FC100 5-way 5-shotaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy