Research map / Large language models / Retrieval and knowledge
Data Wrangling Task Automation Using Code-Generating Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an LLM system that generates executable Python code for data wrangling (imputation, error detection, correction), choosing memory-dependent or memory-independent workflows with iterative prompting, cutting LLM calls versus row-wise invocation.
Problem
Data quality enrichment on large tabular datasets (missing value imputation, error detection, error correction) is costly with row-level LLM calls and traditional/deep methods lack semantic understanding or need task-specific training.
Approach
Use a code-generating LLM to translate data patterns into executable code applied to whole tables, selecting relevant columns via histogram-based gradient boosting, retrieving external knowledge via RAG-style similarity, and refining code iteratively with k-fold validation and majority voting over snippets.
Key results
- On Airline, code generation reaches 0.99 imputation accuracy using 20 LLM calls versus 0.97 with 1376 calls for row-wise invocation.
- On BigBasket error detection, code generation reaches 0.98 accuracy with 32 LLM calls versus 0.94 with 1512 row-wise calls.
- Error correction on Airline matches row-wise accuracy (0.98) using 15 calls versus 1376.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Generate code instead of invoking the LLM row-wise | Data-wrangling system · Airline — Imputation | accuracy: 0.97 -> 0.99 | Helped |
| Generate code instead of invoking the LLM row-wise | Data-wrangling system · Airline — Error Detection | accuracy: 0.98 -> 0.98 | No clear effect |
| Generate code instead of invoking the LLM row-wise | Data-wrangling system · Airline — Error Correction | accuracy: 0.98 -> 0.98 | No clear effect |
| Generate code instead of invoking the LLM row-wise | Data-wrangling system · BigBasket — Imputation | accuracy: 0.94 -> 0.94 | No clear effect |
| Generate code instead of invoking the LLM row-wise | Data-wrangling system · BigBasket — Error Detection | accuracy: 0.94 -> 0.98 | Helped |
| Generate code instead of invoking the LLM row-wise | Data-wrangling system · BigBasket — Error Correction | accuracy: 0.92 -> 0.94 | Helped |
Surprising findings
- Code generation slightly exceeds row-wise LLM accuracy on several tasks while using two orders of magnitude fewer calls.
Limitations
- Evaluated only on two datasets (Airline and BigBasket) with a single LLM.
- Memory-dependent workflow and external knowledge base details are not fully specified in the text.
Experimental setup
Datasets: Airline BigBasket Starbucks
Benchmarks: Airline BigBasket
Metrics: accuracy number of LLM calls
Baselines: row-wise LLM invocation
Builds on: retrieval-augmented generation histogram-based gradient boosting
Closest papers by meaning
- Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models (NeurIPS 2024 · similarity 0.83)
- TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Tasks (ICML 2024 · similarity 0.83)
- Large Scale Transfer Learning for Tabular Data via Language Modeling (NeurIPS 2024 · similarity 0.83)
- EPIC: Effective Prompting for Imbalanced-Class Data Synthesis in Tabular Data Classification via Large Language Models (NeurIPS 2024 · similarity 0.83)
- TSDS: Data Selection for Task-Specific Model Finetuning (NeurIPS 2024 · similarity 0.82)
- MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.