Research map / Large language models / Retrieval and knowledge
Fact-Driven Logical Reasoning for Machine Reading Comprehension
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FocalReasoner, which extracts subject-predicate-object fact units from passages and builds a supergraph over them to model sentence-level and entity-level logical reasoning for machine reading comprehension.
Problem
Logical reasoning in machine reading comprehension requires temporary facts and events, not just commonsense entity-aware knowledge, which existing graph-based methods fail to capture.
Approach
Represent knowledge as syntactically extracted argument-predicate-argument fact units, organize them as Levi graphs, and connect them into a hierarchical supergraph (entity-level inside facts, sentence-level across facts), plus a logical fact regularization loss that pulls subject+predicate toward object embeddings.
Key results
- On ReClor test, FocalReasoner with RoBERTa-large reaches 58.9% (+3.3 over RoBERTa baseline 55.6%); with DeBERTa-xlarge it reaches 73.3% test accuracy.
- On LogiQA test with DeBERTa-xlarge, FocalReasoner reaches 63.0%, outperforming LReasoner (62.7%) and HGN (62.7%).
- On MuTual test, FocalReasoner achieves R4@1 72.7%, versus 71.3% for RoBERTa.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FOCAL REASONER instead of the strongest listed baseline | RoBERTa-large logical reasoning QA · ReClor Test | accuracy: 62.4 -> 58.9 | Hurt |
| Use FOCAL REASONER instead of the strongest listed baseline | DeBERTa-xlarge logical reasoning QA · ReClor Test-H | accuracy: 64.4 -> 63.0 | Hurt |
| Use FOCAL REASONER instead of the strongest listed baseline | DeBERTa-xlarge logical reasoning QA · LogiQA Test | accuracy: 44.2 -> 45.8 | Helped |
| Add a global edge to connect the global node with fact-unit nodes (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 64.6 -> 66.8 | Helped |
| Add co-reference edges between coreferential fact-unit arguments (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 64.8 -> 66.8 | Helped |
| Add logical fact regularization (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 64.2 -> 66.8 | Helped |
| Use multiple edge types instead of a single edge type (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 63.7 -> 66.8 | Helped |
| Add option interactions (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 65.5±0.52 -> 66.8±0.13 | Helped |
| Use fact units instead of named entities only (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 62.8±0.26 -> 66.8±0.13 | Helped |
| Use fact units instead of semantic role labeling only (ablation) | FOCAL REASONER · ReClor Dev | accuracy: 62.2±0.32 -> 66.8±0.13 | Helped |
| Replace the question-option global-node representation with question-only representation (ablation) | FOCAL REASONER · ReClor Dev | accuracy | Hurt |
Surprising findings
- Replacing fact units with named entities or semantic role labels substantially lowers accuracy, suggesting syntactic fact units carry more logical information than commonsense entities.
Limitations
- Performance drops when the number of fact units is large, indicating difficulty with complex logical structures.
- Relies on off-the-shelf syntactic parsing (SpaCy) and coreference resolution, so extraction errors propagate.
Experimental setup
Datasets: ReClor LogiQA MuTual
Benchmarks: ReClor LogiQA MuTual
Metrics: accuracy R4@1 R4@2 MRR
Baselines: RoBERTa DAGN LReasoner MERIt HGN DeBERTa
Builds on: RGCN LReasoner DAGN TransE Levi graph
Closest papers by meaning
- Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension (AAAI 2025 · similarity 0.86)
- Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidance (NeurIPS 2024 · similarity 0.86)
- RULEBREAKERS: Challenging LLMs at the Crossroads between Formal Logic and Human-like Reasoning (ICML 2025 · similarity 0.85)
- Reasoning-as-Logic-Units: Scaling Test-Time Reasoning in Large Language Models Through Logic Unit Alignment (ICML 2025 · similarity 0.85)
- TACT: Advancing Complex Aggregative Reasoning with Information Extraction Tools (NeurIPS 2024 · similarity 0.84)
- COLD: Causal reasOning in cLosed Daily activities (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.