Research map / Multi-agent systems and games / Cooperative multi-agent RL

Adaptive Context Length Optimization with Low-Frequency Truncation for Multi-Agent Reinforcement Learning

NeurIPS 2025 · Multi-agent systems and games › Cooperative multi-agent RL · approach: Cooperative MARL algorithms · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ACL-LFT, a MARL framework where a central agent adaptively selects the context length via RL over Fourier low-frequency truncated histories, with a theorem on adaptive vs fixed context length advantage.

Problem

Fixed large context lengths in deep MARL cause limited exploration efficiency, redundant information, and high computation in long-term dependency and non-Markovian environments.

Approach

A central agent (trained with advantage-based policy gradient) chooses among context lengths at each step, using a discrete-Fourier low-frequency truncation of historical states as its input and an attention-weighted reward from decentralized agents; decentralized agents then act on the selected history plus current state.

Key results

• On the no-cross-agent-history SMACv2 ablation, ACL-LFT reaches 78.9 on 3s5z_vs_3s6z versus 76.1 for AMAGO and 72.6 for Mamba.
• Fixed-length comparison on GRF shows ACL-LFT outperforms fixed context lengths 8, 16, 32, and 64; the best fixed length was 16 steps on 3 vs 1 with Keeper and 8 steps on Counterattack-Hard.
• Ablations show removing adaptive context length hurts most, and removing low-frequency truncation also degrades performance.

Measured effects reported in the paper

Surprising findings

• Longer context lengths do not necessarily improve performance; excessive history is reported to add noise.
• Theorem-based regret bound relies on many strong assumptions (Gaussian posterior, Lipschitz policy, finite latent alphabet), so the theoretical claim is weakly grounded.

Limitations

• Maximum context length is small (4 to 64 steps) and tied to environment episode length.
• Theorem relies on strong, loosely justified assumptions such as Gaussian posteriors and finite latent alphabets.
• Reported gains are on a limited set of environments with a single backbone (MAPPO) in most experiments.

Experimental setup

Datasets: Sample Spread (PettingZoo) MiniGrid Soccer Game Google Research Football Academy 3 vs 1 with Keeper Google Research Football Academy Counterattack-Hard SMACv2 3s5z_vs_3s6z SMACv2 5m_vs_6m SMACv2 corridor

Benchmarks: Sample Spread MiniGrid Soccer Game Academy 3 vs 1 with Keeper Academy Counterattack-Hard SMACv2 3s5z_vs_3s6z SMACv2 5m_vs_6m SMACv2 corridor

Metrics: episode reward win rate exploration efficiency

Baselines: Transformer ToST AMAGO Mamba QMIX QPLEX MAPPO

Builds on: MAPPO Littlewood-Paley theory Transformer AMAGO

Closest papers by meaning

• Balancing Context Length and Mixing Times for Reinforcement Learning at Scale (NeurIPS 2024 · similarity 0.88)
• Value-Based Deep Multi-Agent Reinforcement Learning with Dynamic Sparse Training (NeurIPS 2024 · similarity 0.87)
• HyperMARL: Adaptive Hypernetworks for Multi-Agent RL (NeurIPS 2025 · similarity 0.86)
• ComaDICE: Offline Cooperative Multi-Agent Reinforcement Learning with Stationary Distribution Shift Regularization (ICLR 2025 · similarity 0.86)
• Reidentify: Context-Aware Identity Generation for Contextual Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.86)
• Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.86)