Research map / Large language models / AI and society
AgentExpt: Automating AI Experiment Design with LLM-based Resource Retrieval Agent
Yu Li, Lehui Li, Lin Chen, Qingmin Liao, Fengli Xu, Yong Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add collective perception (citation-context summary) to the retriever representation (ablation) | AgentExpt retriever · Paper-Baseline recommendation | Recall@20: 0.3716 -> 0.4523 | Improved |
| add collective perception (citation-context summary) to the retriever representation (ablation) | AgentExpt retriever · Paper-Dataset recommendation | Recall@20: 0.2493 -> 0.3001 | Improved |
| add first-person self-description to the retriever representation (ablation) | AgentExpt retriever · Paper-Baseline recommendation | Recall@20: 0.4197 -> 0.4523 | Improved |
| add first-person self-description to the retriever representation (ablation) | AgentExpt retriever · Paper-Dataset recommendation | Recall@20: 0.2709 -> 0.3001 | Improved |
| add reasoning supervision to the reranker (ablation) | AgentExpt reranker · Paper-Baseline recommendation | HitRate@5: 0.4284 -> 0.5933 | Improved |
| add reasoning supervision to the reranker (ablation) | AgentExpt reranker · Paper-Dataset recommendation | HitRate@5: 0.3641 -> 0.4557 | Improved |
| add interaction chains as evidence to the reranker (ablation) | AgentExpt reranker · Paper-Baseline recommendation | HitRate@5: 0.5245 -> 0.5933 | Improved |
| add interaction chains as evidence to the reranker (ablation) | AgentExpt reranker · Paper-Dataset recommendation | HitRate@5: 0.4126 -> 0.4557 | Improved |
| add the reasoning-augmented reranker after the retriever (ablation) | AgentExpt pipeline · Paper-Baseline recommendation | HitRate@10: 0.6757 -> 0.6938 | Improved |
| add the reasoning-augmented reranker after the retriever (ablation) | AgentExpt pipeline · Paper-Dataset recommendation | HitRate@10: 0.5308 -> 0.5549 | Improved |
| fine-tune the reranker on interaction chains instead of off-the-shelf GPT-4o (ablation) | AgentExpt reranker · Paper-Baseline recommendation | HitRate@10: 0.6501 -> 0.6938 | Improved |
| fine-tune the reranker on interaction chains instead of off-the-shelf GPT-4o (ablation) | AgentExpt reranker · Paper-Dataset recommendation | HitRate@10: 0.5213 -> 0.5549 | Improved |
Closest papers by meaning
- MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research (NeurIPS 2025 · similarity 0.90)
- AgentSelect: Benchmark for Narrative Query-to-Agent Recommendation (ICML 2026 · similarity 0.89)
- AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite (ICLR 2026 · similarity 0.89)
- InnovatorBench: Evaluating Agents’ Ability to Conduct Innovative AI Research (ICLR 2026 · similarity 0.89)
- From Reproduction to Replication: Evaluating Research Agents with Progressive Code Masking (ICLR 2026 · similarity 0.89)
- MARS: Modular Agent with Reflective Search for Automated AI Research (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
