Research map / Reinforcement learning / Visual and representation RL

HaM-World: Soft-Hamiltonian World Models with Selective Memory for Planning

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: RL for combinatorial routing · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HaM-World, a world model with a q/p/c latent split, Soft-Hamiltonian dynamics on (q,p), and Mamba selective memory as history input, used for CEM planning on six DMC tasks.

Problem

Imagined rollouts in learned world models become unstable as planning horizon grows or dynamics shift, and monolithic latents entangle configuration, momentum, and task semantics while ignoring partial observability.

Approach

Combines history-conditioned Mamba selective memory as input to a shared latent dynamics with a soft Hamiltonian vector field on a canonical (q,p) subspace plus a context subspace c, so the planner uses one latent for prediction, reward/value, and CEM search.

Key results

• Avg. AUC of 117.9 on four-task core suite, +9.5% over TD-MPC2 (107.7).
• Rollout latent MSE averaged over k in {3,5,7} is 1.82 vs 4.02 for TD-MPC2, winning 11/12 cells, but overtaken between k=7 and k=20.
• Highest absolute return in all 12 OOD perturbations, e.g. +10.2% over TD-MPC2 on Finger Spin and +13.6% on Reacher Easy.

Surprising findings

• Rollout error advantage reverses at longer horizons (k=10-20), with crossovers at k=7 to k=20 depending on task.
• Normalized OOD retention does not favour HaM-World; DreamerV3 and TD-MPC2 show higher retention ratios on some tasks.

Limitations

• Evaluation limited to six state-based DMC tasks; no pixel or manipulation experiments.
• Rollout advantage confined to planning-relevant horizon; error grows faster beyond k=7.
• Hamiltonian structure is a soft local bias under forward Euler, not a globally Hamiltonian or identified mechanical model.

Experimental setup

Datasets: DeepMind Control Suite

Benchmarks: DeepMind Control Suite Reacher Easy Finger Spin Cheetah Run Cartpole Swingup Walker Run Hopper Hop

Metrics: final return AUC latent rollout MSE OOD return

Baselines: PPO SAC DreamerV3 TD-MPC2

Builds on: Mamba TD-MPC2 DreamerV3 Hamiltonian Neural Networks JEPA

Closest papers by meaning

• Simplifying Latent Dynamics with Softly State-Invariant World Models (NeurIPS 2024 · similarity 0.89)
• Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Model (CVPR 2026 · similarity 0.88)
• PcLast: Discovering Plannable Continuous Latent States (ICML 2024 · similarity 0.87)
• Predictive but Not Plannable: RC-aux for Latent World Models (NeurIPS 2026 · similarity 0.87)
• Sparse Planning in Visual World Models via Cost Gradients (NeurIPS 2026 · similarity 0.87)
• Agent Planning with World Knowledge Model (NeurIPS 2024 · similarity 0.86)