Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Semantic HELM: A Human-Readable Memory for Reinforcement Learning
Fabian Paischer, Thomas Adler, Markus Hofmarcher, Sepp Hochreiter
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add semantic CLIP-token memory (SHELM) over memory-less PPO | PPO · Avalon 10M steps | mean human normalized score: 0.146 -> 0.155 | No clear effect |
| SHELM vs HELMv2 on Avalon | HELMv2 · Avalon 10M steps | mean human normalized score: 0.145 -> 0.155 | No clear effect |
| train memory-less PPO 50M steps (own) vs published PPO | PPO · Avalon 50M steps | mean human normalized score: 0.165 -> 0.200 | Improved |
| memory-less PPO vs memory-based Dreamerv2 at 50M | PPO · Avalon 50M steps | mean human normalized score: 0.200 vs 0.199 | No clear effect |
| replace semantic tokens with arbitrary retrieval after batch norm (HELMv2-Ret) (ablation) | SHELM · MiniGrid-Memory | accumulated reward | Hurt |
| replace CNN current-observation branch with CLIP encoder (SHELM-CLIP) (ablation) | SHELM · MiniGrid-Memory | accumulated reward | Hurt |
| use task-specific CNN features in history branch (SHELM-CNN) (ablation) | SHELM · MiniGrid-Memory | accumulated reward | Hurt |
| replace pretrained TrXL LM with LSTM on text tokens (SHELM-LSTM) (ablation) | SHELM · MiniGrid-Memory | accumulated reward | Hurt |
| swap ViT-B/16 CLIP encoder for RN50 (ablation) | SHELM · MiniGrid-Memory | accumulated reward | Hurt |
| use designed prompts such as 'A screenshot of a tok' instead of 'An image of a tok' (ablation) | SHELM token retrieval · Avalon and Psychlab | token retrieval quality | Improved |
| use prompts on other environments (ablation) | SHELM token retrieval · MiniGrid/MiniWorld | token retrieval quality | No clear effect |
| SHELM vs HELMv2 on Psychlab continuous recognition | HELMv2 · Psychlab CR | reward | Improved |
Closest papers by meaning
- MemGen: Weaving Generative Latent Memory for Self-Evolving Agents (ICLR 2026 · similarity 0.90)
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.89)
- MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.89)
- Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks (ICML 2026 · similarity 0.89)
- Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents (ICLR 2026 · similarity 0.88)
- EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
