Research map / Large language models / Agents, code and math
TSNBench: Benchmarking LLM Proficiency in Time-Sensitive Networking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TSNBench, a benchmark of 939 expert-validated MCQs and 100 open-ended worst-case delay (WCD) tasks each for CBS and CQF, and evaluates 16 LLMs, finding high MCQ accuracy but large WCD estimation errors.
Problem
No benchmark evaluates LLM proficiency in Time-Sensitive Networking (TSN), where errors in worst-case delay estimation can violate safety-critical real-time constraints.
Approach
Pairs expert-reviewed MCQs generated from TSN research papers with open-ended WCD computation tasks whose ground truth comes from a Network Calculus solver (CBS) and a closed-form bound (CQF), showing MCQ accuracy does not predict computational accuracy.
Key results
- MCQA accuracy ranges from 67.4% (Llama 3.2 1B) to 95.3% (Claude Sonnet 4.5), with GPT-5 at 95.0%.
- GPT-5 achieves the best CBS WCD MAPE of 36.2%, while most models exceed 80%.
- On CQF, Mistral Large 3 achieves the lowest MAPE at 41.8%, while GPT-4o has the lowest median MAE (1.2 µs).
Surprising findings
- o3 has the highest ECE (0.1874) yet the lowest confidently-wrong rate (3.4%).
- Reasoning models did not outperform the best non-reasoning models on MCQA.
Limitations
- Open-ended evaluation covers only CBS and CQF, not TAS or other mechanisms.
- MCQs derived only from open-access research papers, not IEEE standards.
- Open-ended tasks are closed-book zero-shot without NC equations in the prompt.
Experimental setup
Datasets: 83 open-access TSN research papers IEEE 802.1 TSN literature
Benchmarks: TSNBench MCQA TSNBench open-ended CBS WCD TSNBench open-ended CQF WCD
Metrics: accuracy consistency ECE Brier score confidently wrong rate MAE MAPE median MAE
Baselines: GPT-5 o3 Claude Sonnet 4.5 Gemini 2.5 Flash Mistral Large 3 Llama 3.2 1B
Builds on: MMLU MMLU-Pro TeleQnA 6G-Bench NetConfEval
Closest papers by meaning
- A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability (NeurIPS 2026 · similarity 0.84)
- Dr.LLM: Dynamic Layer Routing in LLMs (ICLR 2026 · similarity 0.84)
- Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMs (ICML 2024 · similarity 0.83)
- ProtocolBench: Which LLM MultiAgent Protocol to Choose? (ICML 2026 · similarity 0.83)
- MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability (NeurIPS 2024 · similarity 0.83)
- On Calibration of LLM-based Guard Models for Reliable Content Moderation (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.