Research map / Large language models / Agents, code and math
Pushing the Limits of Rule Reasoning in Transformers through Natural Language Satisfiability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on hard instances sampled near the SAT critical region instead of naive sampling | T5 GRL 10var · GRL hard evaluation subset | accuracy: 8 points | Helped |
| Train on hard instances sampled near the SAT critical region instead of biased easy instances | T5 GRL 10var · GRL hard evaluation subset | accuracy: 8 points | Helped |
| Evaluate standard RuleTaker-trained models on hard-sampled RuleTaker theories | RT-T5 · RuleTaker to Hard RT | accuracy: 97.5 -> 57.7 | Hurt |
| Evaluate standard RuleTaker-trained models on hard-sampled RuleTaker theories | RT-RoBERTa · RuleTaker to Hard RT | accuracy: 98.7 -> 59.6 | Hurt |
| Train on smaller-size GRL problems and evaluate on larger problems | T5 GRL 5,12 · GRL-eval 20-50 variables | accuracy: 80.7; 72.7; 61.4; 51.8 | Hurt |
Closest papers by meaning
- Can Transformers Reason Logically? A Study in SAT Solving (ICML 2025 · similarity 0.92)
- Transformer Encoder Satisfiability: Complexity and Impact on Formal Reasoning (ICLR 2025 · similarity 0.90)
- When can transformers reason with abstract symbols? (ICLR 2024 · similarity 0.90)
- The Expressive Power of Transformers with Chain of Thought (ICLR 2024 · similarity 0.90)
- Grokking of Implicit Reasoning in Transformers: A Mechanistic Journey to the Edge of Generalization (NeurIPS 2024 · similarity 0.90)
- Transformers Provably Solve Parity Efficiently with Chain of Thought (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.