Research map / Large language models / Retrieval and knowledge
SR-FoT: A Syllogistic-Reasoning Framework of Thought for Large Language Models Tackling Knowledge-based Reasoning Tasks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SR-FoT, a five-stage prompting framework that guides LLMs through syllogistic deductive reasoning (question explanation, major premise, minor premise questions and answers, final syllogism) with restricted per-stage visibility, improving accuracy on ScienceQA, StrategyQA, and BoolQ.
Problem
LLM reasoning with chain-of-thought often does not follow strict deductive reasoning, so its conclusions can lack rigor on knowledge-based reasoning QA tasks.
Approach
Mimics human syllogistic reasoning by having the LLM autonomously generate a major premise from built-in knowledge and a minor premise via self-posed and self-answered minor premise questions, then derive the answer, while limiting each stage to only its necessary preceding information.
Key results
- On ScienceQA with DeepSeek-V2, SR-FoT reaches 91.8% vs 87.0% Base and 88.6% CoT; SC-SR-FoT reaches 93.0%.
- On BoolQ with GPT-3.5-turbo, SC-SR-FoT reaches 85.0% vs 82.8% for SC-CoT.
- In a manual rigor analysis of 50 GPT-3.5-turbo cases per dataset, SR-FoT's rigor rate on BoolQ was 0.96 vs 0.80 for CoT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SR-FoT instead of CoT | GPT-3.5-turbo · ScienceQA | accuracy: 86.9 -> 87.4 | Helped |
| Use SR-FoT instead of CoT | GPT-3.5-turbo · StrategyQA | accuracy: 91.3 -> 92.1 | Helped |
| Use SR-FoT instead of CoT | GPT-3.5-turbo · BoolQ | accuracy: 79.4 -> 81.7 | Helped |
| Add self-consistency sampling and aggregation to SR-FoT | GPT-3.5-turbo · BoolQ | accuracy: 82.8 -> 85.0 | Helped |
| Use SR-FoT instead of CoT | DeepSeek-V2 · ScienceQA | accuracy: 88.6 -> 91.8 | Helped |
| Use SR-FoT instead of CoT | DeepSeek-V2 · BoolQ | accuracy: 86.0 -> 87.3 | Helped |
| Add self-consistency sampling and aggregation to SR-FoT | DeepSeek-V2 · StrategyQA | accuracy: 92.4 -> 95.0 | Helped |
| Use SR-FoT instead of CoT | Qwen1.5-32B-Chat · StrategyQA | accuracy: 91.0 -> 91.9 | Helped |
| Add self-consistency sampling and aggregation to SR-FoT | Qwen1.5-32B-Chat · BoolQ | accuracy: 84.3 -> 88.2 | Helped |
| Add the question explanation stage (ablation) | SR-FoT on DeepSeek-V2 · ScienceQA | accuracy: 90.4 -> 91.8 | Helped |
| Add the major premise (ablation) | SR-FoT on DeepSeek-V2 · ScienceQA | accuracy: 90.9 -> 91.8 | Helped |
| Add the minor premise (ablation) | SR-FoT on DeepSeek-V2 · ScienceQA | accuracy: 87.7 -> 91.8 | Helped |
Surprising findings
- Under DeepSeek-V2 and Qwen1.5-32B-Chat, CoT sometimes performs worse than the Base prompt on the three datasets.
- Directly generating major and minor premises in one stage ('all in one stage') underperforms the multi-stage design on ScienceQA with DeepSeek-V2 (87.7 vs 91.8 in Table 4).
Limitations
- Does not guarantee rigorous syllogistic reasoning; errors arise from major/minor premise generation and final reasoning.
- Evaluation of rigor relies on manual analysis of small samples (50 per dataset).
- Multi-round self-consistency variant requires 10 samples per question, increasing cost.
Experimental setup
Datasets: ScienceQA StrategyQA BoolQ
Benchmarks: ScienceQA StrategyQA BoolQ
Metrics: accuracy rigor rate error source proportion
Baselines: Base CoT SC-CoT C-CoT
Builds on: Chain-of-Thought Self-consistency CoT Complexity-based CoT Least-to-Most
Closest papers by meaning
- TypedThinker: Diversify Large Language Model Reasoning with Typed Thinking (ICLR 2025 · similarity 0.89)
- Reasoning Scaffolding: Distilling the Flow of Thought from LLMs (ICLR 2026 · similarity 0.89)
- Decompose, Analyze and Rethink: Solving Intricate Problems with Human-like Reasoning Cycle (NeurIPS 2024 · similarity 0.88)
- THOUGHT PROPAGATION: AN ANALOGICAL APPROACH TO COMPLEX REASONING WITH LARGE LANGUAGE MODELS (ICLR 2024 · similarity 0.88)
- Large Language Models Meet Symbolic Provers for Logical Reasoning Evaluation (ICLR 2025 · similarity 0.88)
- RATT: A Thought Structure for Coherent and Correct LLM Reasoning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.