Research map / Large language models / Language, safety and interpretability

How Do Language Models Understand Tables? A Mechanistic Analysis of Cell Location

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Mechanistically dissects how LLMs locate table cells via activation patching, probing, and steering, finding a three-stage pipeline (semantic binding, coordinate localization via delimiter counting, information extraction) and reuse of the same heads for multi-cell queries.

Problem

LLMs must process linearized two-dimensional tables, but the internal mechanisms by which they understand table structure, specifically for the atomic cell location task, remain unexplained.

Approach

Applies activation patching, linear probing, attention-head ablation, and RoPE-based query-key analysis to a synthetic cell-location task to decompose table understanding into three stages, showing column indices are counted via pipe delimiters, lie in a linear subspace steerable by vector arithmetic, and that multi-cell queries reuse the same heads.

Key results

• Ablating top-20 row alignment heads drops Qwen3-4B exact-match accuracy from 94.4% to 5.2% on atomic cell location.
• Column and row linear probes peak at R^2 of 0.954 and 0.809 at layer 17; delimiter-token column-index heads achieve Effect Score 0.958 versus 0.025 for cell-token heads.
• Ablating row alignment heads on multi-row queries drops accuracy from 81.8% to 0.4%, indicating reuse of the same heads.

Surprising findings

• Column alignment head ablation barely hurts performance, and column localization appears to operate without strong semantic attention weights.
• Delimiter-token probes reach high R^2 in early layers, before cell-content probes, suggesting coordinates are resolved at delimiters and then propagated to cells.

Limitations

• Analysis relies on synthetic tables with randomized entity values, which may not reflect real-world tables.
• Primary mechanistic analysis is on Qwen3-4B; other models are reported in appendix only.
• Predicted indices diverge from ground truth as table size grows, indicating degradation at larger dimensions.

Experimental setup

Datasets: synthetic tables generated by the authors (500 atomic, 500 multi-row, 500 multi-column samples)

Benchmarks: synthetic atomic cell location benchmark (500 samples) synthetic multi-row and multi-column location benchmarks (500 samples each)

Metrics: accuracy effect score (normalized logit difference) R^2 of linear probes top-1 alignment accuracy error-type distribution

Builds on: activation patching indirect object identification circuit analysis RoPE

Closest papers by meaning

• NeedleInATable: Exploring Long-Context Capability of Large Language Models towards Long-Structured Tables (NeurIPS 2025 · similarity 0.88)
• TableRAG: Million-Token Table Understanding with Language Models (NeurIPS 2024 · similarity 0.87)
• MMTU: A Massive Multi-Task Table Understanding and Reasoning Benchmark (NeurIPS 2025 · similarity 0.85)
• HeGTa: Leveraging Heterogeneous Graph-enhanced Large Language Models for Few-shot Complex Table Understanding (AAAI 2025 · similarity 0.85)
• CellVerse: Do Large Language Models Really Understand Cell Biology? (NeurIPS 2025 · similarity 0.84)
• RADAR: Benchmarking Language Models on Imperfect Tabular Data (NeurIPS 2025 · similarity 0.84)