Research map / Large language models / Retrieval and knowledge
UQE: A Query Engine for Unstructured Databases
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Universal Query Engine (UQE), which runs SQL-like queries with natural-language predicates over unstructured tables by combining statistical sampling, online active learning, and a compiler-style planner to cut LLM calls.
Problem
Analytics (counting, retrieval, grouping) over large collections of unstructured data such as images, dialogs and reviews is costly because LLMs must be run over every row, and long-context LLMs and RAG do not handle aggregation over whole databases well.
Approach
Treats LLM predicates as expensive oracles: uses stratified importance sampling over multimodal embeddings for unbiased aggregation estimates and online active learning with a cheap surrogate for recall-oriented retrieval, orchestrated by a SQL-style compiler that plans clause order and fusion so only a budgeted number of rows reach the LLM.
Key results
- On IMDB conditional count, UQE with claude-3-haiku reaches 5.75% relative error versus 13.67% for UDF and 49.02% for lc-gpt-4-turbo, at about $0.01 per query versus $0.37.
- Semantic retrieval on IMDB reaches F1 0.978 for UQE versus 0.875 for MIPS and 0.397 for lc-gpt-4-turbo.
- UQE with budget 128 matches the quality of the UDF baseline at budget 512, roughly a 16x compute reduction on IMDB aggregation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UQE instead of UDF for conditional aggregation | IMDB · IMDB, sentiment_positive | relative error: 13.67% ± 6.24% -> 5.75% ± 3.43% | Helped |
| Use UQE instead of UDF for conditional aggregation | ABCD · ABCD, account_access and single_item_query | relative error: 18.99% ± 9.85% -> 11.75% ± 9.78%; 26.95% ± 22.16% -> 12.32% ± 10.53% | Helped |
| Use UQE instead of UDF for conditional aggregation | AirDialog · AirDialog, book, no_flight, and no_reservation | relative error: 10.15% ± 7.64% -> 4.98% ± 2.26%; 21.08% ± 16.78% -> 8.78% ± 8.12%; 21.19% ± 12.10% -> 7.23% ± 5.40% | Helped |
| Use UQE with GPT-4o instead of UDF-gpt-4o | Clevr · Clevr, # spheres > 3 | relative error: 19.35% ± 13.81% -> 15.14% ± 10.71% | Helped |
| Use UQE instead of MIPS | Semantic retrieval · ABCD, account_access | F1: 0.961 -> 0.940 ± 0.019 | Hurt |
| Use UQE instead of MIPS | Semantic retrieval · AirDialog, no_reservation | F1: 0.965 -> 0.969 ± 0.004 | Helped |
| Use UQE instead of MIPS | Semantic retrieval · IMDB and remaining ABCD, AirDialog, and Clevr conditions | F1 | Helped |
| Use UQE instead of lc-claude-3-opus | Abstraction and aggregation · AirDialog, ABCD account_access, and ABCD single_item_query | EMD: 0.121 ± 0.014 -> 0.111 ± 0.019; 0.113 ± 0.010 -> 0.110 ± 0.016; 0.011 ± 0.006 -> 0.005 ± 0.002 | Helped |
| Replace stratified sampling with uniform random sampling (ablation) | UQE aggregation sampling · Aggregation queries over three text datasets | estimation variance | Hurt |
| Replace online learning with uniform random sampling (ablation) | UQE non-aggregation retrieval · Semantic retrieval queries | recall | Hurt |
| Reduce retrieval budget from 256 to 128 (ablation) | UQE · IMDB semantic retrieval | F1: 0.978 -> 0.974 | Hurt |
| Reduce retrieval budget from 128 to 64 (ablation) | UQE · IMDB semantic retrieval | F1: 0.974 -> 0.921 | Hurt |
Surprising findings
- Long-context lc-LLM baselines performed worse than the sampling-based UDF baseline on conditional counting despite seeing more data.
- A single GPT-4o prompted per image with manual aggregation gave error similar to lc-LLM baselines, while UQE with the same model halved worst-case error.
Limitations
- Only single-table queries are supported; joins and other aggregations such as MAX are not handled by the estimator.
- Retrieval latency is high (tens of seconds on a laptop CPU) due to online surrogate retraining.
- Hyperparameters were set by educated guess and not tuned; results depend on LLM backend and the evaluation uses small budgets (128-256 rows).
Experimental setup
Datasets: IMDB ABCD AirDialog Clevr AudioMNIST
Benchmarks: IMDB ABCD AirDialog Clevr AudioMNIST
Metrics: relative error F1 score earth mover's distance cost per query latency
Baselines: lc-gpt-4-turbo lc-claude-3-opus UDF MIPS lc-gpt-4o UDF-gpt-4o Gemini Pro 1.5
Builds on: SQL importance sampling Bayesian optimization RAG
Closest papers by meaning
- Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows (ICLR 2025 · similarity 0.85)
- Structure-Guided Large Language Models for Text-to-SQL Generation (ICML 2025 · similarity 0.84)
- Holistic Reasoning with Long-Context LMs: A Benchmark for Database Operations on Massive Textual Data (ICLR 2025 · similarity 0.84)
- UDA: A Benchmark Suite for Retrieval Augmented Generation in Real-World Document Analysis (NeurIPS 2024 · similarity 0.84)
- Alpha-SQL: Zero-Shot Text-to-SQL using Monte Carlo Tree Search (ICML 2025 · similarity 0.83)
- SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.