Research map / Vision-language and multimodal / Visual question answering
RETQA: A Large-Scale Open-Domain Tabular Question Answering Dataset for Real Estate Sector
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces RETQA, a 20,762-QA-pair open-domain Chinese tabular QA dataset for real estate with SLU (intent/slot) labels, and SLUTQA, an ICL framework that uses SLU labels to improve table retrieval and answer generation.
Problem
Automated question answering over real estate tabular data is limited by a lack of specialized open-domain tabular QA datasets, and existing benchmarks do not cover long tables, multi-table queries, and open-domain retrieval together.
Approach
Augment tabular QA with spoken language understanding labels (intent and slot) that guide LLM-generated query summaries for BM25 table retrieval and filter irrelevant rows/columns or generate SQL for answering, without fine-tuning the LLM.
Key results
- SLUTQA (FT) raises Table EM over vanilla by 3.89, 4.35, and 3.88 points for Qwen2-7B, GLM4-9B, and Qwen2-72B on markdown answers.
- For SQL answers, SLUTQA (FT) with Qwen2-72B reaches pass@1 of 82.71% versus 51.40% vanilla.
- Fine-tuned BERT reaches 99.33 intent F1 and 99.42 slot F1, above few-shot LLM ICL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SLUTQA with ICL instead of vanilla answering | Qwen2 7B · RETQA markdown answers | Table EM (%): 15.05 -> 16.65 | Helped |
| Use SLUTQA with ICL instead of vanilla answering | GLM4 9B · RETQA markdown answers | Table EM (%): 4.73 -> 9.15 | Helped |
| Use SLUTQA with ICL instead of vanilla answering | Qwen2 72B · RETQA markdown answers | Table EM (%): 11.14 -> 14.81 | Helped |
| Use SLUTQA with fine-tuned BERT for SR instead of vanilla answering | Qwen2 7B · RETQA SQL answers | ECR (%): 60.11 -> 87.28 | Helped |
| Use SLUTQA with fine-tuned BERT for SR instead of vanilla answering | Qwen2 72B · RETQA SQL answers | pass@1: 51.40 -> 82.71 | Helped |
| Use SLUTQA with fine-tuned BERT for SR instead of vanilla answering | GLM4 9B · RETQA SQL answers | pass@1: 34.52 -> 66.81 | Helped |
| Replace vanilla query summaries with SR summaries generated using SLU labels (ablation) | Qwen2 7B · RETQA table retrieval | F1: 91.42 -> 94.03 | Helped |
| Replace vanilla query summaries with SR summaries generated using SLU labels (ablation) | Qwen2 72B · RETQA table retrieval | F1: 96.76 -> 97.50 | Helped |
| Replace vanilla query summaries with SR summaries generated using SLU labels (ablation) | GLM4 9B · RETQA table retrieval | F1: 92.87 -> 93.92 | Helped |
| Add SLU labels to prompts for table simplification and answer generation (ablation) | Qwen2 7B · RETQA markdown gold-table answers | Cell EM F1: 35.87 -> 41.71 | Helped |
| Add SLU labels to prompts for table simplification and answer generation (ablation) | GLM4 9B · RETQA markdown gold-table answers | Cell EM F1: 48.80 -> 56.85 | Helped |
| Add SLU labels to prompts for table simplification and answer generation (ablation) | Qwen2 72B · RETQA markdown gold-table answers | Cell EM F1: 65.59 -> 68.26 | Helped |
Surprising findings
- Vanilla outperforms the Simplified (no SLU label) table-simplification method on Qwen2-7B, suggesting unguided simplification removes important information.
Limitations
- Experiments do not include multiple runs, variance, or significance tests, as acknowledged by the authors.
- Dataset is Chinese only; English version promised but not yet released.
- Queries are template-based and rewritten by an LLM, so they may not fully reflect real user queries.
Experimental setup
Datasets: RETQA
Benchmarks: RETQA
Metrics: Table EM Row/Column/Cell EM precision/recall/F1 ECR pass@1 table retrieval precision/recall/F1 intent and slot F1
Baselines: Vanilla ICL BM25 top-1 retrieval Simplified table ICL
Builds on: BERT BM25 OPENTAB TAP4LLM StructGPT
Closest papers by meaning
- TableBench: A Comprehensive and Complex Benchmark for Table Question Answering (AAAI 2025 · similarity 0.87)
- Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries (AAAI 2024 · similarity 0.85)
- Accurate and Regret-Aware Numerical Problem Solver for Tabular Question Answering (AAAI 2025 · similarity 0.85)
- RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content (NeurIPS 2024 · similarity 0.84)
- SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs (ICLR 2024 · similarity 0.84)
- TrustUQA: A Trustful Framework for Unified Structured Data Question Answering (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.