Research map / Architectures and efficiency / RNNs and normalization
FLAME: A Small Language Model for Spreadsheet Formulas
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FLAME, a 60M-parameter T5-style encoder-decoder trained only on Excel formulas with sketch-based deduplication, an Excel-specific tokenizer, and domain-specific pre-training objectives, evaluated on formula repair, completion, and retrieval against much larger code models.
Problem
Large code LLMs are expensive to train and deploy for spreadsheet formula assistance, even though formula-only tasks such as repair and completion need only a narrow language. The paper asks whether a much smaller model trained only on Excel formulas can compete.
Approach
Train a small T5-small-based model exclusively on deduplicated Excel formulas, using an Excel lexer-aware tokenizer and two new objectives (language-aware masked span prediction and user-inspired noise) alongside tail masking and random noise denoising.
Key results
- On last-mile repair with fine-tuning, FLAME reaches 0.89 top-5 on the Forum benchmark, matching Davinci (0.89) and beating Cushman (0.88), while its top-1 of 0.76 is second to Cushman's 0.79.
- In zero-shot completion at a 90% prefix, FLAME achieves 0.70 exact match and 0.84 sketch match, above Davinci (0.64 EM, 0.76 SM).
- For formula retrieval, FLAME reaches Pearson 0.904 after fine-tuning, versus 0.899 for CodeT5, and embeds formulas about 41% faster than CodeT5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FLAME rather than Codex-Cushman for zero-shot Forum top-1 repair | Formula repair models · Forum last-mile repair | top-1 exact match: 0.55 -> 0.71 | Helped |
| Use FLAME rather than CodeT5 for formula completion | Formula completion models · Formula completion, 0.90 prefix | top-5 exact match: 0.27 -> 0.70 | Helped |
| Reduce model size from 60M to 16M parameters (ablation) | FLAME · Formula completion, 0.90 prefix | exact match: 0.70 -> 0.59 | Hurt |
| Replace per-workbook deduplication with global formula deduplication (ablation) | FLAME · Formula completion, 0.90 prefix | exact match: 0.70 -> 0.45 | Hurt |
| Add language-aware masked span prediction (laMSP) (ablation) | T5 formula model · Forum formula completion, 0.90 prefix | exact match: 0.22 -> 0.51 | Helped |
| Add user-inspired denoising to laMSP (ablation) | T5 formula model · Forum last-mile repair | top-1 exact match: 0.73 -> 0.75 | Helped |
| Add user-inspired denoising to laMSP (ablation) | T5 formula model · Forum formula completion, 0.90 prefix | exact match: 0.51 -> 0.51 | No clear effect |
| Add Excel-specific tokenizer to laMSP and user-inspired denoising (ablation) | T5 formula model · Forum formula completion, 0.90 prefix | exact match: 0.51 -> 0.70 | Helped |
| Add Excel-specific tokenizer to laMSP and user-inspired denoising (ablation) | T5 formula model · Forum last-mile repair | top-1 exact match: 0.75 -> 0.76 | Helped |
Surprising findings
- Global deduplication across workbooks, which yields a much smaller 591K corpus, reduces performance by up to 30 points, so per-workbook deduplication was preferred.
- Fine-tuning Codex-Cushman and FLAME for completion degraded performance, possibly due to overfitting, so zero-shot was used.
Limitations
- Gains over larger models are mixed: FLAME trails Cushman at top-1 on the Forum repair benchmark and on some completion sketch-match settings.
- Evaluated only on Excel formulas and the three tasks studied; does not use the spreadsheet data context.
- Codex baselines are few-shot or zero-shot with a fixed prompt, and the completion benchmark is small (273 formulas).
Experimental setup
Datasets: Excel formulas from 1.8M public web workbooks mrexcel.com and excelforum.com help forum formulas Enron spreadsheet corpus
Benchmarks: AAAI 2021 last-mile repair benchmark (273 forum formulas) Synthetic repair benchmark Formula completion benchmark (273 formulas) Enron formula retrieval (1000 formulas)
Metrics: top-1 exact match top-5 exact match exact match sketch match Pearson correlation latency
Baselines: Codex-Cushman Codex-Davinci CodeT5 CodeBERT GraphCodeBERT
Builds on: T5 BART Mixture-of-Denoisers byte pair encoding LoRA
Closest papers by meaning
- SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation (NeurIPS 2024 · similarity 0.85)
- OpenTab: Advancing Large Language Models as Open-domain Table Reasoners (ICLR 2024 · similarity 0.83)
- OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text (ICLR 2024 · similarity 0.83)
- Regress, Don't Guess: A Regression-like Loss on Number Tokens for Language Models (ICML 2025 · similarity 0.82)
- EnterpriseBench: Evaluating LLM Agents on End-to-End Spreadsheet Tasks in Finance (NeurIPS 2026 · similarity 0.82)
- Composing Biases by Using CP to Decompose Minimal Functional Dependencies for Acquiring Complex Formulae (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.