atlas

Research map / Large language models / AI and society

Resource Democratization: Is Compute the Binding Constraint on AI Research?

AAAI 2024 · Large language models › AI and society · approach: Social-science view of AI evaluation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Survey of over 500 U.S. AI researchers (academic and industry) finding that talent and data limit research plans more than compute, with little academia-industry difference in compute use, though heavy compute users want more.

Problem

Whether compute access is the binding constraint on AI research, and how compute resource stratification between academia and industry affects researchers, had not been directly studied by asking researchers themselves.

Approach

Surveys AI researchers directly about project inputs, compute use, research plan changes, and views on a National AI Research Resource, comparing academic and industry subgroups and compute-use levels rather than relying on model-cost estimates or practitioner interviews.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Prioritize talent rather than large amounts of compute for project successAI researchers’ most significant projects · Most significant project successvery or extremely important (%): 90% vs 52%Mixed
Compare insufficient data with insufficient compute as a reason to change project plansAI researchers · Rejecting, revising, and abandoning projectsfrequency of project-plan changes: all p < .001Mixed
Compare researcher-availability limitations with insufficient compute as a reason to change project plansAI researchers · Rejecting and abandoning projectsfrequency of project-plan changes: both p < .001Mixed
Shift expectations from compute toward better algorithms as drivers of AI progressAI researchers’ expectations for the next decade · AI progress over the next decadestrong agreement (%): More compute: 59% -> 40%; Better algorithms: 31% -> 53%Mixed
Compare academic and industry researchers’ GPU-hour useAI researchers in academia versus industry · Most compute-intensive project in the preceding five yearsGPU-hours: p = .200No clear effect
Compare academic and industry researchers’ compute spendingAI researchers in academia versus industry · Most compute-intensive project in the preceding five yearscompute spending: p < .001Mixed
Compare academic and industry researchers’ concern about future compute accessAI researchers in academia versus industry · Future compute accessconcernNo clear effect
Use more compute for the most compute-intensive projectAI researchers · Concern about sufficient compute for future researchfuture compute concernHelped

Surprising findings

Limitations

Experimental setup

Datasets: survey responses from AI researchers (410 complete, 123 partial) Web of Science author records from 20 AI venues 2016-2021 Revelio Labs LinkedIn data

Metrics: Likert-scale mean ratings percent of respondents chi-squared test of independence Mann-Whitney U test with Bonferroni correction

Builds on: National AI Research Resource proposal

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy