Research map / Large language models / Agents, code and math
AMOR: A Recipe for Building Adaptable Modular Knowledge Agents Through Process Feedback
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AMOR, a finite-state-machine-based modular knowledge agent built on LLaMA-2 chat models with MA-MoE, warmed up on public QA data and adapted via process feedback optimized with KTO.
Problem
Building knowledge-intensive language agents that have precise, controllable reasoning logic, can adapt to specific domains, and can be steered by human feedback on intermediate reasoning steps rather than only final outcomes.
Approach
Model agent reasoning as a finite state machine over disentangled LLM and tool modules, train module-specific FFN experts (MA-MoE) on automatically constructed warm-up data, and adapt via process-level feedback (per-module right/wrong or refinements) optimized with KTO plus MLE regularization.
Key results
- AMOR with process feedback on LLaMA-2-13B-Chat reaches 48.6 EM / 55.3 F1 on HotpotQA, versus 45.8 / 54.9 for the 7B variant and 40.8 EM for outcome-feedback adaptation on 7B.
- Without fine-tuning, AMOR on GPT-4 reaches 55.2 EM on HotpotQA, improving over ReAct by about 41% on average across base LLMs as reported by the authors.
- Human feedback adaptation on HotpotQA (L-7B) improves EM from 45.8 (silver) to 50.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FSM-based AMOR reasoning instead of ReAct | LLaMA-2-7B-Chat without fine-tuning · HotpotQA | EM: 12.2 -> 26.0 | Helped |
| add warm-up fine-tuning (ablation) | AMOR · HotpotQA | EM: 26.0 -> 33.6 | Helped |
| add warm-up fine-tuning (ablation) | AMOR · PubMedQA | ACC: 62.9 -> 73.4 | Helped |
| add warm-up fine-tuning (ablation) | AMOR · Qasper | EM: 4.5 -> 11.1 | Helped |
| use process feedback instead of outcome feedback for adaptation (ablation) | AMOR · HotpotQA | EM: 40.8 -> 45.8 | Helped |
| use process feedback instead of outcome feedback for adaptation (ablation) | AMOR · PubMedQA | ACC: 77.5 -> 81.1 | Helped |
| use process feedback instead of outcome feedback for adaptation (ablation) | AMOR · Qasper | EM: 9.4 -> 19.1 | Helped |
| replace SFT with KTO (ablation) | AMOR Process with MA-MoE · HotpotQA | EM: 43.2 -> 45.8 | Helped |
| replace SFT with KTO (ablation) | AMOR Process with MA-MoE · PubMedQA | ACC: 79.3 -> 81.1 | Helped |
| replace SFT with KTO (ablation) | AMOR Process with MA-MoE · Qasper | EM: 18.6 -> 19.1 | Helped |
| add module-specific awareness to the MoE architecture (ablation) | AMOR with SFT · HotpotQA | EM: 41.6 -> 43.2 | Helped |
| add module-specific awareness to the MoE architecture (ablation) | AMOR with SFT · PubMedQA | ACC: 78.8 -> 79.3 | Helped |
Surprising findings
- Outcome feedback adaptation gave negligible gains in evidence recall compared with warm-up only, while process feedback substantially improved recall.
- A standard MoE without module-aware routing performed comparably to a plain Transformer.
Limitations
- Experiments use silver feedback automatically derived from gold answers and evidence; human feedback studied only on HotpotQA with one annotator.
- Decompose and Complete modules show lower accuracy after adaptation, indicating room for improvement.
- Focused on text corpora as knowledge bases; extension to structured knowledge is future work.
Experimental setup
Datasets: 2WikiMultiHopQA Musique NaturalQuestions BoolQ HotpotQA PubMedQA Qasper
Benchmarks: HotpotQA PubMedQA Qasper
Metrics: EM F1 ACC recall of gold evidence
Baselines: CoT OneR ReAct ReWoo Lumos AgentLM FireAct Self-RAG
Builds on: Chain-of-thought ReAct Self-RAG Contriever KTO Mixture-of-Experts
Closest papers by meaning
- A$^2$FM: An Adaptive Agent Foundation Model for Tool-Aware Hybrid Reasoning (ICLR 2026 · similarity 0.87)
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.87)
- RefactorBench: Evaluating Stateful Reasoning in Language Agents Through Code (ICLR 2025 · similarity 0.87)
- DyFlow: Dynamic Workflow Framework for Agentic Reasoning (NeurIPS 2025 · similarity 0.87)
- From an LLM Swarm to a PDDL-empowered Hive: Planning Self-executed Instructions in a Multi-modal Jungle (ICLR 2025 · similarity 0.86)
- Reinforce LLM Reasoning through Multi-Agent Reflection (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.