AI Paper Atlas

Research map / Large language models / AI and society

HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face

Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, Yueting Zhuang

NeurIPS 2023 · Large language models › AI and society · approach: Data commons and human baselines

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use GPT-3.5 instead of Vicuna-7b as controllerHuggingGPT · single task planningaccuracy: 23.86 -> 52.62Improved
use GPT-3.5 instead of Vicuna-7b as controllerHuggingGPT · sequential task planningF1: 22.89 -> 51.92Improved
use GPT-3.5 instead of Vicuna-7b as controllerHuggingGPT · graph task planningGPT-4 Score: 19.17 -> 50.48Improved
use GPT-4 instead of GPT-3.5 as controllerHuggingGPT · human-annotated sequential planningaccuracy: 18.18 -> 41.36Improved
use GPT-4 instead of GPT-3.5 as controllerHuggingGPT · human-annotated graph planningF1: 16.45 -> 49.28Improved
use GPT-3.5 instead of Vicuna-13b as controllerHuggingGPT · human evaluation, 130 requestssuccess rate (%): 15.64 -> 63.08Improved
increase demonstration variety from 2 to 10 task types (ablation)HuggingGPT with GPT-3.5 · single task planningaccuracy: 43.31 -> 52.83Improved
increase demonstration variety from 2 to 10 task types (ablation)HuggingGPT with GPT-4 · single task planningaccuracy: 65.59 -> 67.52Improved
increase demonstration variety from 2 to 10 task types (ablation)HuggingGPT with GPT-3.5 · graph task planningF1: 43.42 -> 64.24Improved
increase demonstration variety from 2 to 10 task types (ablation)HuggingGPT with GPT-4 · sequential task planningedit distance (%): 47.17 -> 39.32Improved
increase demonstration variety from 2 to 6 task types (ablation)HuggingGPT with GPT-3.5 · sequential task planningF1: 32.15 -> 43.19Improved
increase number of demonstrations beyond 4 (ablation)HuggingGPT · task planningaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy