Research map / Large language models / Agents, code and math

WirelessMathBench-XL: A Contamination-Audited Benchmark for Wireless Mathematical Reasoning

NeurIPS 2026 · Large language models › Agents, code and math · approach: Mathematical and structural reasoning benchmarks

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Releases WirelessMathBench-XL, a 4,027-problem wireless math benchmark from arXiv papers, with a reverse-probe 13-gram RedPajama-arXiv contamination audit, cleaned subsets, and GRPO-trained WirelessMathLM learnability baselines.

Problem

Benchmarks built from arXiv papers can overlap public pretraining text, and released technical benchmarks often lack the provenance and metadata needed to audit that overlap at problem level.

Approach

A reverse-probe lexical-overlap audit that indexes benchmark prompts at a fixed 13-gram threshold and streams public corpora (RedPajama-arXiv), emitting per-problem hit metadata and a strict zero-hit subset S0, with memory scaling with the benchmark rather than the corpus; bundled with source-paper provenance and a bounded full-vs-S0 stability summary.

Key results

• Filtering to the strict zero-hit subset S0 (3,853 problems, 95.7%) changes every evaluated model's accuracy by less than 1 pp; frontier 16k rows cluster between 86.5% and 91.3%.
• Only 30/800 test items carry detected 13-gram overlap; the all-flagged-correct counterfactual bounds positive score inflation at 0.31-0.51 pp for frontier rows.
• WirelessMathLM-7B with GRPO reaches 47.88% on the locked 2k test split vs 41.00% for DeepSeekMath-7B-RL and 40.00% for Qwen2.5-Math-7B-Instruct.

Surprising findings

• The only systematic nonzero full-vs-S0 deltas occur in WirelessMathLM rows, in the direction opposite to what a lexical-overlap shortcut would predict.
• Expert inter-rater agreement is low (Cohen's kappa 0.164), so the expert screening is treated as provenance rather than a reliability guarantee.

Limitations

• Audit covers only exact prompt-surface 13-gram overlap with RedPajama-arXiv; paraphrase, target-answer, post-training and closed-corpus exposure are not covered.
• With only 30/800 flagged items the full-vs-S0 comparison is structurally underpowered and cannot detect inflation above about 0.31-0.51 pp.
• The v1.0 split is problem-level rather than source-paper-disjoint (766 of 800 test items share a source paper with training), and only 1,542 records are released due to licensing.

Experimental setup

Datasets: WirelessMathBench-XL training split RedPajama-arXiv OpenWebMath Dolma v1.7 arxiv ProofPile-2 arxiv Math12K

Benchmarks: WirelessMathBench-XL test split (800 items) WirelessMathBench-XL public test subset (310 items)

Metrics: accuracy paired-bootstrap delta (full minus S0) zero-hit rate proxy flag rate

Baselines: DeepSeekMath-7B-RL Qwen2.5-Math-7B-Instruct GPT-5-mini GPT-5.2-chat Gemini-2.5-Pro Gemini-2.5-Flash Grok-4-Fast DeepSeek-V3.1 Claude-4.0-Sonnet o4-mini

Builds on: GRPO WirelessMathBench RedPajama DeepSeek-R1

Closest papers by meaning

• LiveBench: A Challenging, Contamination-Limited LLM Benchmark (ICLR 2025 · similarity 0.87)
• Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.87)
• A Careful Examination of Large Language Model Performance on Grade School Arithmetic (NeurIPS 2024 · similarity 0.85)
• RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics (NeurIPS 2025 · similarity 0.85)
• GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (ICLR 2025 · similarity 0.85)
• MathArena: Evaluating LLMs on Uncontaminated Math Competitions (NeurIPS 2025 · similarity 0.85)