AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Walrus: A Cross-domain Foundation Model for Continuum Dynamics

Michael McCabe, Payel Mukhopadhyay, Tanya Marwah, Bruno Régaldo-Saint Blancard, François Rozet, Cristiana Diaconu, Lucas Thibaut Meyer, Kaze W. K. Wong, Mohammad-Hadi Sotoudeh, Alberto Bietti, Irina Espejo Morales, Rio Alexa Fear and 13 more

ICML 2026 · Architectures and efficiency › RNNs and normalization · approach: Neural scaling laws

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add patch jittering during autoregressive rollout (ablation)Walrus · pretraining datasets (19), long-horizon rolloutmedian VRMSE: 1.07 -> 0.34 (Maze)Improved
add patch jittering (ablation)Walrus · pretraining datasets (19)unstable rollouts (VRMSE >= 1): 10 -> 3 of 19 datasetsImproved
add patch jittering (ablation)Walrus · TRL (2D) full trajectorymedian VRMSE: 76.9 -> 0.75Improved
add patch jittering (ablation)Walrus · MultiQuadrantsP full trajectorymedian VRMSE: 0.60 -> 0.62No clear effect
add patch jittering (ablation)Walrus · Active Matter full trajectorymedian VRMSE: 0.98 -> 0.98No clear effect
use topology-aware sampling plus adaptive tokenization and differential batch sizes (ablation)Walrus training · distributed training throughputthroughput: +262% over naive FSDPImproved
normalize token counts across datasets with adaptive-compute tokenization (ablation)Walrus training · distributed training throughputthroughputHurt
use Walrus instead of prior foundation models (MPP, Poseidon, DPOT)downstream finetuning · downstream tasks one-stepVRMSE: 63.6% average reductionImproved
use Walrus instead of prior foundation modelsdownstream finetuning · downstream tasks medium-rangeVRMSE: 48.3% average reductionImproved
use Walrus instead of DPOT on linear acoustic tasks at long horizonsDPOT-H · Staircase avg T in [21:60]VRMSE: 0.0031 vs 0.0074Hurt
add augmentation (2D to 3D embedding, time strides) in pretraining HalfWalrus (ablation)HalfWalrus · pretraining metricspretraining lossHurt
add augmentation (2D to 3D embedding, time strides) in pretraining HalfWalrus (ablation)HalfWalrus · unseen downstream tasksVRMSEImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy