Research map / Multi-agent systems and games / LLM agents

MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MoRSE, a task-oriented multi-agent system that decomposes tasks into a subtask DAG and uses a prototype-routed mixture of role and subtask LoRA experts trained with hierarchical GRPO (two-layer credit assignment) on code-generation benchmarks.

Problem

Prompt-level role differentiation in LLM multi-agent systems yields redundant, correlated agent outputs and limited specialization, and existing parameter-adaptive MAS RL methods adapt only at the coarse role level; joint optimization of experts and router under sparse task rewards is also unstable.

Approach

Combine a dependency-aware subtask DAG with (role, subtask)-conditioned LoRA experts selected by a prototype-based semantic router, and optimize them with hierarchical GRPO that uses a within-route advantage for experts and a cross-route advantage for the router.

Key results

• On SRDD Test split with Qwen3-4B, MoRSE reaches 86.25% Exec and 0.360 ECI Product vs 65.00% and 0.286 for the base model.
• On SciCode Test split with Qwen3-4B, MoRSE reaches 29.00% Step Pass and 10.00% Problem Pass, versus 21.43% and 0.00% for the base model.
• Ablation on Qwen3-4B: swapping standard GRPO for HGRPO with MoLE raises SRDD Exec from 73.75% to 86.25% and SciCode Step Pass from 23.00% to 29.00%.

Surprising findings

• Plugging MoLE into standard GRPO without hierarchical credit is fragile, underperforming the fixed-LoRA variant on SRDD Exec (73.75% vs 82.50%).
• Post-training uplift is smaller on the strongest backbone (Gemma-4-31B), and the untrained framework sometimes falls below the single-agent baseline on the held-out split.

Limitations

• Evaluation limited to code-generation benchmarks and smaller backbones.
• Only two roles (execute, merge) and a conservative subtask-expert pool size; most results from a single seed.
• SRDD training reward and ECI metric share component sources, a possible evaluation confound.

Experimental setup

Datasets: SRDD SciCode

Benchmarks: SRDD SciCode

Metrics: Exec (%) ECI Mean ECI Product Step Pass (%) Mean Step Pass (%) Problem Pass (%)

Baselines: Base model ChatChain MacNet AFlow MoRSE w/o MoLE (fixed LoRA + GRPO) MoRSE w/o HGRPO (MoLE + standard GRPO)

Builds on: GRPO LoRA MacNet AFlow ChatChain

Closest papers by meaning

• AOrchestra: Automating Sub-Agent Creation for Agentic Orchestration (ICML 2026 · similarity 0.90)
• Improving the Efficiency of Language Agent Teams with Adaptive Task Graphs (NeurIPS 2026 · similarity 0.89)
• MoRAgent: Parameter Efficient Agent Tuning with Mixture-of-Roles (ICML 2025 · similarity 0.88)
• AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents (ICLR 2026 · similarity 0.88)
• OWL: Optimized Workforce Learning for General Multi-Agent Assistance in Real-World Task Automation (NeurIPS 2025 · similarity 0.87)
• Autonomous Agents for Collaborative Task under Information Asymmetry (NeurIPS 2024 · similarity 0.87)