Research map / Multi-agent systems and games / LLM agents

ForecastCompass: Guiding Agentic Forecasting with Adaptive Factor Memory

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FoCo, a hierarchical, factor-centric verbal memory for agentic forecasting that is revised by contrasting forecast and retrospective trajectories, improving Brier and ECE on Prophet Arena and FutureX.

Problem

Generic agent memories (trajectories, reflections, factual graphs) do not capture reusable predictive factors and calibration knowledge needed for forecasting unresolved events under incomplete evidence.

Approach

A taxonomy-indexed memory per subcategory with two components, predictive-factor memory and calibration-oriented reasoning memory, revised iteratively by contrasting original forecasting trajectories with post-resolution retrospective trajectories while avoiding event-level hindsight.

Key results

• On GPT-5-mini, FoCo reaches average Brier 0.075 on Prophet Arena vs 0.150 for Base and 0.109 for A-Mem, and 0.187 on FutureX vs 0.241 for Base.
• On Gemini-2.5-Flash, FoCo reaches average Brier 0.118 on Prophet Arena vs 0.202 for Base, and ECE 0.090 vs 0.106.
• Bootstrap test on Prophet Arena shows FoCo's Brier improvement over Base of -0.0738 (95% CI [-0.0923, -0.0560], p<0.001).

Surprising findings

• Simply adding retrospective trajectories to Reflexion gives only a small gain (FutureX Brier 0.203 to 0.198), while FoCo's structured memory gives much larger gains.
• The Base (Retro) reference, which uses post-resolution evidence, is not an upper bound and is outperformed by FoCo on Prophet Arena Brier with GPT-5-mini? No: Base (Retro) (0.109) is better than FoCo (0.075) only on none; FoCo beats it on Prophet Arena Brier for Gemini? Not reported as such.

Limitations

• Small evaluation sets (640 and 242 questions over five weeks) with a single short evaluation window
• Memory is longer than baselines (about 2.2K tokens on average), and gains may partly reflect added structure rather than length alone
• Performance depends on epoch selection using a single development week

Experimental setup

Datasets: Prophet Arena FutureX

Benchmarks: Prophet Arena FutureX

Metrics: Brier score expected calibration error

Baselines: Base Base (Retro) FoCo (Static) Mem0 Reflexion A-Mem Graphiti Reflexion + Retro Trajectory

Builds on: Reflexion Mem0 A-Mem Graphiti

Closest papers by meaning

• MemGen: Weaving Generative Latent Memory for Self-Evolving Agents (ICLR 2026 · similarity 0.87)
• TimeClaw: A Time-Series AI Agent with Exploratory Execution Learning (NeurIPS 2026 · similarity 0.87)
• AdaMEM: Test-Time Adaptive Memory for Language Agents (ICML 2026 · similarity 0.87)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.86)
• Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks (ICML 2026 · similarity 0.86)
• AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.86)