Research map / Multi-agent systems and games / LLM agents

Multi-Stage Prompting for Next Best Agent Recommendations in Adaptive Workflows

AAAI 2024 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a three-stage LLM prompting pipeline (task-to-agent mapping, task-specific next-best-agent ranking, and agent-specific input/output matching) that recommends next best agents at cold-start from agent metadata and process knowledge.

Problem

Recommending the next best agent (API) to a business user in adaptive, ad-hoc workflows from conversational utterances when no historical execution data exists (cold-start), with hundreds of agents exceeding LLM prompt limits.

Approach

A staged prompting approach in which an LLM first narrows the agent set per process task, then ranks top-k next agents for the target agent and task, and finally re-ranks globally by checking whether the target agent's output semantically matches the candidate's input, with guidance-based controlled generation and post-processing for grounding.

Key results

• MPT-7B achieved the highest task-to-agent mapping average recall (0.912 on ABCD, 0.86 on Recruitment), ahead of OpenLLaMA-13B and Flan-T5-XXL.
• On ABCD, the framework reached NBA accuracy 0.5125 and agent recall 0.844 for task-specific NBAs, versus 0.243 and 0.384 for the best baseline (ChromaDB).
• On the Recruitment dataset, the framework reached NBA accuracy 0.464 and agent recall 0.694, versus 0.4047 and 0.583 for the best baseline (Elasticsearch).

Measured effects reported in the paper

Surprising findings

• Elasticsearch keyword search was the strongest baseline on the Recruitment dataset because agent descriptions have high keyword overlap.

Limitations

• Evaluation uses only two datasets, one of which is proprietary and ABCD was manually transformed into agent format.
• Human-evaluator mappings show low recall, and stage-3 accuracy is low (0.21-0.35 NBA accuracy).
• No historical data is used, so the approach cannot leverage usage logs that could improve recommendations.

Experimental setup

Datasets: ABCD Recruitment Process (IBM proprietary)

Benchmarks: ABCD Recruitment Process

Metrics: recall precision NBA accuracy agent recall

Baselines: NBA using BERT ChromaDB semantic search Elasticsearch keyword search

Builds on: Toolformer Gorilla BM25

Closest papers by meaning

• AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation (ICML 2026 · similarity 0.86)
• Optimize Once, Execute Fast: Latency-Aware Multi-Agent Workflow Learning for Recurrent Queries (NeurIPS 2026 · similarity 0.86)
• AgentRecBench: Benchmarking LLM Agent-based Personalized Recommender Systems (NeurIPS 2025 · similarity 0.85)
• RecFlow: An Industrial Full Flow Recommendation Dataset (ICLR 2025 · similarity 0.83)
• WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models (ICLR 2025 · similarity 0.83)
• UNEX-RL: Reinforcing Long-Term Rewards in Multi-Stage Recommender Systems with UNidirectional EXecution (AAAI 2024 · similarity 0.83)