AI Paper Atlas

Research map / Large language models / Agents, code and math

VERIFY: A Novel Multi-Domain Dataset Grounding LTL in Contextual Natural Language via Provable Intermediate Logic

Paapa Kwesi Quansah, Pablo Rivas, Ernest Bonnah

ICLR 2026 · Large language models › Agents, code and math · approach: Inductive and abstract reasoning evaluation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use ITL instead of LTL as the source for NL generationLlama-3-8B-Instruct (FT) · VERIFYBERTScore F1: 0.91 -> 0.94Improved
use ITL instead of LTL as the source for NL generationLlama-3-8B-Instruct (FT) · VERIFYROUGE-L F1: 0.67 -> 0.73Improved
use ITL instead of LTL as the source for NL generationT5-large · VERIFYBERTScore F1: 0.67 -> 0.89Improved
add domain and activity context to NL-to-LTL training (ablation)Llama-3-8B-Instruct (FT) · VERIFY NL-to-LTLsemantic equivalence (%): 7.7 -> 28.2Improved
add domain and activity context to NL-to-LTL training (ablation)Llama-3-8B-Instruct (FT) · VERIFY NL-to-LTLexact match (%): 0.8 -> 4.1Improved
add domain and activity context to NL-to-ITL training (ablation)Llama-3-8B-Instruct (FT) · VERIFY NL-to-ITLsemantic equivalence (%): 13.9 -> 41.5Improved
increase LTL formula AST depth from 1-4 to 13+Llama-3-8B-Instruct (FT) · VERIFY NL-to-LTLsemantic equivalence (%): 35.4 -> 15.2Hurt
hold out a domain from training (Aerospace) (ablation)Llama-3-8B-Instruct (FT) · VERIFY NL-to-LTLsemantic equivalence (%): 27.5 -> 19.2Hurt
hold out a domain from training (Home Automation) (ablation)Llama-3-8B-Instruct (FT) · VERIFY NL-to-LTLsemantic equivalence (%): 30.1 -> 22.5Hurt
hold out a domain from training (Financial) (ablation)Llama-3-8B-Instruct (FT) · VERIFY NL-to-LTLsemantic equivalence (%): 25.8 -> 16.7Hurt
use code-focused DeepSeek-Coder instead of Llama-3-8Bfine-tuned LLM · VERIFY NL-to-LTLsemantic equivalence (%): 28.2 -> 31.5Improved
use fine-tuned LLM instead of T5-largeT5-large · VERIFY NL-to-LTLsemantic equivalence (%): 22.3 -> 28.2Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy