Research map / Large language models / Agents, code and math
StreamBench: Towards Benchmarking Continuous Improvement of Language Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces StreamBench, an online benchmark that streams tasks with binary correctness feedback to measure whether LLM agents improve over time, and proposes streaming baselines (Self-StreamICL, MAM-StreamICL) that share memory across multiple LLM agents.
Problem
Existing benchmarks measure LLMs' innate out-of-the-box capabilities and do not assess whether LLM agents continuously improve from an input-feedback stream after deployment.
Approach
Simulate an online streaming setting over seven datasets with binary correctness feedback, and propose memory-based streaming ICL baselines: Self-StreamICL stores only self-generated correct input-output pairs, and MAM-StreamICL round-robins several LLM agents over a shared memory at the cost of a single agent.
Key results
- MAM-StreamICL achieves the best averaged results across all seven datasets, e.g., 83.50 on DDXPlus vs 52.85 zero-shot (averaged over three LLMs).
- Self-StreamICL with gpt-4o-2024-08-06 improves DDXPlus accuracy to 92.01 from 70.64 zero-shot and BIRD to 42.63 from 34.42.
- Using only incorrect past self-outputs can degrade performance below zero-shot, while using only correct outputs consistently helps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Self-StreamICL instead of zero-shot prompting | LLM agents · Spider, CoSQL, BIRD, DS-1000, ToolBench, DDXPlus, HotpotQA | task metric | Helped |
| share memory across GPT-3.5, Gemini-1.0-Pro, and Claude-3-Haiku agents with round-robin inference | Self-StreamICL · DDXPlus | accuracy: 66.16, 69.50, 76.02 -> 83.50 | Helped |
| retrieve only correct past examples instead of all past triples (ablation) | GrowPrompt and MemPrompt · StreamBench tasks | performance | Helped |
| retrieve only incorrect past examples instead of all past triples (ablation) | GrowPrompt and MemPrompt · StreamBench tasks | performance | Hurt |
| use single-agent memory instead of shared multi-agent memory (ablation) | MAM-StreamICL · DDXPlus | accuracy: 83.50 -> 65.31; 83.50 -> 72.05; 83.50 -> 81.52 | Hurt |
| use all-MiniLM-L6-v2 text encoder instead of bge-base-en-v1.5 (ablation) | Self-StreamICL · DDXPlus | accuracy: 66.16 -> 63.61; 76.02 -> 78.91; 86.34 -> 83.50 | Mixed |
| use bge-small-en-v1.5 text encoder instead of bge-base-en-v1.5 (ablation) | Self-StreamICL · DDXPlus | accuracy: 66.16 -> 63.55; 76.02 -> 75.51; 86.34 -> 83.90 | Mixed |
| apply Self-StreamICL to BIRD sequences with distributional shifts (ablation) | Self-StreamICL · BIRD | execution accuracy: 35.07 -> 36.31; 33.25 -> 32.60; 37.61 -> 36.57; 41.20 -> 40.48; 42.63 -> 43.09 | Mixed |
Surprising findings
- Informing agents only of incorrect past outputs (verbalized negative feedback) often fails to help and can hurt performance below zero-shot.
- Self-StreamICL, which stores only self-generated correct pairs, outperforms GrowPrompt and MemPrompt that store all feedback.
Limitations
- Limited to text tasks; does not cover image or audio modalities.
- Simulated binary correctness feedback may not capture the noise and diversity of real-world feedback (sim2real gap).
- Uses only cost-effective API models in main experiments; high cost limited full seed robustness runs.
Experimental setup
Datasets: Spider CoSQL BIRD DS-1000 ToolBench DDXPlus HotpotQA
Benchmarks: Spider CoSQL BIRD DS-1000 ToolBench DDXPlus HotpotQA
Metrics: execution accuracy pass@1 accuracy exact match
Baselines: Zero-shot Few-shot Chain-of-thought Self-Refine GrowPrompt MemPrompt
Builds on: MemPrompt Self-Refine Reflexion ExpeL GrowPrompt
Closest papers by meaning
- AgentBench: Evaluating LLMs as Agents (ICLR 2024 · similarity 0.89)
- The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement (NeurIPS 2025 · similarity 0.88)
- LOCA-bench: Benchmarking Language Agents Under Controllable and Extreme Context Growth (ICML 2026 · similarity 0.88)
- FT-Dojo: Towards Autonomous LLM Fine-Tuning with Language Agents (ICML 2026 · similarity 0.87)
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.87)
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.