Research map / Multi-agent systems and games / LLM agents
NADER: Neural Architecture Design via Multi-Agent Collaboration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NADER, a GPT-4o-driven multi-agent framework (Reader, Proposer, Modifier, Reflector) that iteratively modifies architectures as graphs to design novel networks beyond NAS-Bench-201 search spaces.
Problem
Neural architecture design is costly and existing NAS is limited to predefined search spaces; prior LLM-based NAD methods work independently without learning from feedback or past experience and rely on code representations that distract from architecture design.
Approach
A team of LLM agents (research team: Reader and Proposer; development team: Modifier and Reflector) iteratively modifies a base network, with a Reflector that learns from immediate verification feedback and a retrieval database of design experience, and a DAG-based text graph representation instead of code.
Key results
- NADER (ResNet init, 500 architectures) reaches 76.00% CIFAR-100 test accuracy, above the NAS-Bench-201 optimal of 73.51%.
- On the NAD benchmark for micro-level modifications, the full system reaches 0.92 executability, 0.96 quality and 0.88 success rate, versus 0.49 success rate without graph representation or agents' feedback.
- With random init and 10 architectures, NADER reaches 49.63% test accuracy on ImageNet16-120 versus 31.02% for LeMo-NADe-Gemini.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NADER with ResNet initialization and 500 architecture designs instead of the NAS-Bench-201 optimum | NADER · CIFAR-100 test | accuracy: 73.51 -> 76.00 | Helped |
| Use NADER with random initialization and 10 designs instead of the NAS-Bench-201 optimum | NADER · ImageNet16-120 test | accuracy: 47.31 -> 49.63±0.80 | Helped |
| Add both Reader and Proposer to the research team (ablation) | NADER · ImageNet16-120 test | accuracy: 47.14±1.30 -> 48.72±0.14 | Helped |
| Add Reader while retaining Proposer (ablation) | NADER research team · CIFAR-10 test | accuracy: 93.73±0.38 -> 94.20±0.38 | Hurt |
| Add Proposer while retaining Reader (ablation) | NADER research team · CIFAR-10 test | accuracy: 91.72±2.52 -> 94.20±0.38 | Helped |
| Use graph representation instead of direct code generation (ablation) | NADER development team · NAD benchmark Macro | success rate: 0.40 -> 0.42 | Helped |
| Add learning from immediate feedback (LIF) (ablation) | NADER development team · NAD benchmark Macro | success rate: 0.42 -> 0.54 | Helped |
| Add learning from design experience (LDE) (ablation) | NADER development team · NAD benchmark Macro | success rate: 0.54 -> 0.68 | Helped |
| Add learning from design experience (LDE) (ablation) | NADER development team · NAD benchmark Micro | success rate: 0.62 -> 0.88 | Helped |
| Add learning from immediate feedback (LIF) (ablation) | NADER development team · NAD benchmark Macro | executability: 0.54 -> 0.64 | Helped |
| Add learning from immediate feedback (LIF) (ablation) | NADER development team · NAD benchmark Macro | quality: 0.78 -> 0.84 | Helped |
| Add graph representation and LIF to code-generation baseline (ablation) | NADER development team · NAD benchmark Micro | success rate: 0.49 -> 0.62 | Helped |
Surprising findings
- Code-generation baseline achieved 64% executability but only 63% quality, and the graph representation improved quality while using fewer tokens.
- About 19.2% of the 500 NADER-designed models failed to converge, which the authors suggest can be detected early from loss values.
Limitations
- Relies on GPT-4o API; results depend on a closed proprietary LLM.
- Architectures constrained to the NAS-Bench-201 macro skeleton for fair comparison.
- About 19% of generated models fail to converge.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet16-120 NAS-Bench-201
Benchmarks: NAS-Bench-201 CIFAR-10 CIFAR-100 ImageNet16-120 NAD Benchmark
Metrics: validation accuracy test accuracy executability quality success rate number of tokens
Baselines: ENAS DARTS SETN DSNAS PC-DARTS SNAS iDARTS GDAS DRNAS REA RS REINFORCE
Builds on: EvoPrompting LLMatic GENIUS MetaGPT LeMo-NADe
Closest papers by meaning
- SAGE-NAS: Synergizing LLM-Based Semantic Agent with Graph-Based Evaluator for Neural Architecture Search (ICML 2026 · similarity 0.88)
- RZ-NAS: Enhancing LLM-guided Neural Architecture Search via Reflective Zero-Cost Strategy (ICML 2025 · similarity 0.87)
- NAT: Neural Architecture Transformer for Accurate and Compact Architectures (NeurIPS 2019 · similarity 0.87)
- einspace: Searching for Neural Architectures from Fundamental Operations (NeurIPS 2024 · similarity 0.86)
- Structuring Open-Ended NAS: Semi-Automated Design Knowledge Structuring with LLMs for Efficient Neural Architecture Search (NeurIPS 2026 · similarity 0.86)
- Towards Neural Architecture Search through Hierarchical Generative Modeling (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.