AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

A Benchmark and Framework for Evaluating Next Action Predictions in Spreadsheets

Tejas Agrawal, Vu Le, Sumit Gulwani, Gust Verbruggen

ICML 2026 · Large language models › Language, safety and interpretability · approach: Data and evaluation methodology · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use a stronger LLM (GPT-5 with reasoning vs GPT-5 mini)zero-shot LLM predictor · NAPE single-action repredictUAS (%): 18.0 -> 32.7Improved
fine-tune SmolLM2-360M on synthetic operation sequencesSmolLM2-360M · NAPE single-action repredictUAS (%): 21.7 -> 26.8Improved
fine-tune SmolLM2-135M on synthetic operation sequencesSmolLM2-135M · NAPE single-action repredictUAS (%): 18.3 -> 23.2Improved
accept every non-empty prediction (ALWAYS) instead of GREEDY (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 22.3 -> -19.2Hurt
use a stricter acceptance rule (GREEDY-2 / HYBRID-2) instead of GREEDY (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 22.3 -> 20.3 -> 17.5Hurt
accept on precision >= 90% regardless of utility (P90) instead of GREEDY (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 22.3 -> 17.0Hurt
increase prediction stride from 1 to 8 (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 22.3 -> 9.8Hurt
increase context window from 32 to 128 operations (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 22.3 -> 27.6Improved
increase context window from 128 to 512 or 2048 operations (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 27.6 -> 26.2 (512), 27.4 (2048)No clear effect
cap actions per prediction at 1 instead of unlimited (ablation)GPT-5 multi-action · NAPE multi-actionUAS (%): 22.3 -> 20.3Hurt
disable re-prediction after an accepted action (ablation)GPT-5 single-action · NAPE single-actionUAS (%): 27.4 -> 20.3Hurt
turn off context shortening of long cell values (ablation)GPT-5 · NAPE single-actionUAS (%): 27.4 -> 27.7No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy