AI Paper Atlas

Research map / Large language models / Agents, code and math

fxBench: Evaluating and Understanding Formula Suggestions in Spreadsheets

Sanket Mhatre, Sumit Gulwani, Vu Le, Yasharth Bajpai, Gust Verbruggen

NeurIPS 2026 · Large language models › Agents, code and math · approach: Software engineering agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces fxBench, a benchmark for evaluating proactive formula suggestion in spreadsheets, and studies how well models handle this task.

Problem

Proactive formula suggestions in spreadsheets require collecting and representing the right context for a machine learning model, which is challenging to evaluate; the abstract is truncated, so details are limited.

Approach

A benchmark framing formula suggestion as a proactive task in spreadsheet software (Excel and Google Sheets style), with an emphasis on context collection and representation for the model.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use denser Minisheet representation instead of windowed SpreadsheetCoder with frontier modelGPT 5.2 · fxBench zero-shotaccuracy relative to upper bound: SpreadsheetCoder 89.6 vs Minisheet 48.0Hurt
increase viewport from 20x10 to 40x30 (ablation)windowed context · fxBench context coverageupper-bound coverage: 69.0% -> 89.9%Improved
increase Minisheet border width w beyond 2 (ablation)Minisheet · fxBench context coverageintent printed coverage: 86.3% (w=2) -> 87.5% (w=4)No clear effect
switch table representation format (ablation)GPT 5.2 / Sonnet 4.6 · fxBench zero-shot full sheetaccuracy: GPT 5.2: CSV 49.7 vs pipe-delimited 58.8Mixed
add dependency chain to oracle context (ablation)GPT 5.2 oracle · fxBench oracleaccuracy: +6.2%Improved
add worksheet formulas, same-column formulas or neighbors to oracle context (ablation)GPT 5.2 oracle · fxBench oracleaccuracy: +5.2%, +4.6%, +4.4%Improved
add cell format to oracle context (ablation)GPT 5.2 oracle · fxBench oracleaccuracyNo clear effect
add natural-language intent description to oracle context (ablation)GPT 5.2 oracle · fxBench oracleaccuracy: 66.2 -> 97.2Improved
add all structured signals to oracle context (ablation)GPT 5.2 oracle · fxBench oracle failureswrong formula type failures: wrong type 77 -> 43; wrong ranges 15 -> 16; wrong abstentions 15 -> 26Mixed
supervised fine-tuning on Minisheet with curated data (ablation)Qwen3-8B · fxBench Minisheet contextpass@1: 9.7% -> 21.2%Improved
train on curated data instead of random sample (ablation)Qwen3-8B fine-tuning · fxBench Minisheet contextpass@1: 11.3% -> 21.2%Improved
scale curated training data to 39,161 records (ablation)Qwen3-8B fine-tuning · fxBench Minisheet contextpass@1: 21.2% -> 20.6%No clear effect

Experimental setup

Benchmarks: fxBench

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy