AI Paper Atlas

Research map / Large language models / Agents, code and math

LocalBench: Benchmarking LLMs on County-Level Local Knowledge and Reasoning

Zihan Gao, Yifei Xu, Jacob Thebault-Spieker

AAAI 2026 · Large language models › Agents, code and math · approach: Mathematical and structural reasoning benchmarks · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add web search to GPT-4.1 (ablation)GPT-4.1 · LocalBench non-numerical QAGPT Judge accuracy: 47.0 -> 35.6Hurt
add search grounding to Gemini-2.5-Pro (ablation)Gemini-2.5-Pro · LocalBench non-numerical QAGPT Judge accuracy: 52.5 -> 56.8Improved
add web search to GPT-4.1 (ablation)GPT-4.1 · LocalBench numerical QAnumerical accuracy: 6.2 -> 15.5Improved
add search grounding to Gemini-2.5-Pro (ablation)Gemini-2.5-Pro · LocalBench numerical QAnumerical accuracy: 12.8 -> 12.8Mixed
add web search to GPT-4.1 (ablation)GPT-4.1 · LocalBench non-numerical QAanswer rate: 100.0 -> 92.9Hurt
add search grounding to Gemini-2.5-Pro (ablation)Gemini-2.5-Pro · LocalBench non-numerical QAanswer rate: 100.0 -> 91.7Hurt
scale Qwen3 from 32B dense to 235B-A22B MoEQwen3 · LocalBench non-numerical QAGPT Judge accuracy: 27.7 -> 27.3Hurt
scale Qwen3 from 8B to 14BQwen3 · LocalBench non-numerical QAGPT Judge accuracy: 22.9 -> 27.5Improved
scale Qwen3 from 14B to 32BQwen3 · LocalBench non-numerical QAGPT Judge accuracy: 27.5 -> 27.7No clear effect
use newer GPT-4.1 instead of GPT-4oGPT-4o · LocalBench non-numerical QAGPT Judge accuracy: 32.8 -> 47.0Improved
scale Qwen3 from 8B to 235B-A22B on numerical QAQwen3 · LocalBench numerical QAnumerical accuracy: 3.1 -> 6.6No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy