Research map / Large language models / Agents, code and math
Evaluating Robustness of Reasoning Models on Parameterized Logical Problems
Naïm Es-sebbani, Esteban Marquer, Yakoub Salhi, Zied Bouraoui
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace template verbalizer with LLM-written narratives (ablation) | reasoning models on 2-SAT · 2-SAT diagnostic benchmark | decision accuracy: drops by about 25 points | Hurt |
| add filler clauses at fixed core size (ablation) | reasoning models · EquivalenceCore, Backbone, MonoBridge | accuracy: filler significantly hurts (|C|=75 r=25 vs 75,0) | Hurt |
| shuffle clause order instead of construction order (ablation) | four main models · all generators | accuracy: p<0.003 | Hurt |
| shuffle clause order (ablation) | four main models · EquivalenceCore | accuracy | Improved |
| shuffle clause order (ablation) | four main models · ImplicationCycle | accuracy: p=0.035 | Mixed |
| vary bridge position (ablation) | models · MonoBridge (75 clauses) | accuracy: p=7.788e-1 | No clear effect |
| vary overall variable density (ablation) | models · MonoBridge | accuracy: p=2.073e-7 | Mixed |
| vary positive-part ratio toward balanced split (ablation) | models · MonoBridge (75 clauses) | accuracy: p<0.05 | Hurt |
| increase backbone fraction beta from 0 to 1 (ablation) | models · Backbone (75 clauses) | accuracy: drop of around 10% | Hurt |
| increase monotone density (ablation) | models · Backbone (75 clauses) | accuracy: increase of up to 20% for the worst model | Improved |
| flip polarity pi of monotone part (ablation) | models · Backbone | accuracy: all p-values > 0.9 | No clear effect |
| increase free-to-bound ratio (ablation) | models · EquivalenceCore (20 clauses) | accuracy: p<0.005 | Hurt |
Closest papers by meaning
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning (ICML 2025 · similarity 0.90)
- Putnam-AXIOM: A Functional & Static Benchmark for Measuring Higher Level Mathematical Reasoning in LLMs (ICML 2025 · similarity 0.89)
- Reasoning Structure of Large Language Models (ICML 2026 · similarity 0.88)
- When Reasoning Meets Its Laws (NeurIPS 2026 · similarity 0.88)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.88)
- DiffAdapt: Difficulty-Adaptive Reasoning for Token-Efficient LLM Inference (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
