AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

QATCH: Benchmarking SQL-centric tasks with Table Representation Learning Models on Your Data

Simone Papicchio, Paolo Papotti, Luca Cagliero

NeurIPS 2023 · Large language models › Retrieval and knowledge · approach: Domain-specific QA datasets · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Evaluate TRL models on proprietary tables instead of SpiderTAPAS/TAPEX/OmniTab/ChatGPT · QA-TRLaverage metric: TAPAS 0.51 -> 0.35 (Spider -> E-commerce)Hurt
Evaluate ChatGPT 3.5 on proprietary E-commerce instead of SpiderChatGPT 3.5 · QA-TRLaverage metric: 0.76 -> 0.28Hurt
Use ChatGPT 3.5 instead of TAPAS on Finance tablesChatGPT 3.5 · QA-TRLaverage metric: 0.35 -> 0.46Improved
Use ChatGPT 3.5 instead of TAPAS on E-commerce tablesChatGPT 3.5 · QA-TRLaverage metric: 0.35 -> 0.28Hurt
Switch from QA-TRL to SP-TRL (SQL generation) for ChatGPT 3.5 on E-commerceChatGPT 3.5 · E-commerceaverage metric: 0.28 -> 0.98Improved
Evaluate ChatGPT 3.5 SP-TRL on proprietary Medicine instead of Spider devChatGPT 3.5 · SP-TRLaverage metric: 0.94 -> 0.99No clear effect
Evaluate GAP on proprietary Medicine instead of Spider devGAP · SP-TRLaverage metric: 0.95 -> 0.70Hurt
Evaluate UnifiedSKG on proprietary E-commerce instead of Spider devUnifiedSKG · SP-TRLaverage metric: 0.85 -> 0.76Hurt
Evaluate ChatGPT 3.5 SP-TRL on Spider train instead of Spider devChatGPT 3.5 · SP-TRLaverage metric: 0.94 -> 0.92No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy