Research map / Vision-language and multimodal / Multimodal LLMs

Logic Unseen: Revealing the Logical Blindspots of Vision-Language Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces LogicBench, a 51k-pair VLM logic benchmark across image, video, anomaly, and medical scenarios, and LogicCLIP, which fine-tunes CLIP with logic-aware hard negatives and three losses (contrastive, multiple-choice, logic-structure classification) to reduce logical blindspots.

Problem

CLIP-style vision-language models rely on superficial semantic alignment and fail to distinguish logically different captions (e.g., conjunction vs. disjunction, causality, negation), limiting reliability in practical use.

Approach

Generate logically perturbed hard negative captions with multiple LLMs from parsed MSCOCO captions, then fine-tune CLIP with a combined objective adding a 4-way multiple-choice loss and a logical-category classification loss on top of standard contrastive learning.

Key results

• LogicCLIP-L raises image MCQ accuracy from 36.90% (OpenAI CLIP-L) to 83.93%, and LogicCLIP-B reaches 81.91% MCQ vs 38.56% for CLIP-B on the image scenario.
• LogicCLIP-B improves COCO R@1 from 30.44% to 42.54% over OpenAI CLIP-B.
• Human evaluators score 96.32% on image MCQ, while the best baseline NegFull scores 60.04%.

Surprising findings

• Adding the multiple-choice loss alone gives very high LogicBench MCQ accuracy but collapses general retrieval (COCO R@1 about 0.0016), showing the logic objectives alone destroy general alignment.
• LLaVA 1.5-7B with CLIP-L encoder outperforms CLIP-L on MCQ, suggesting LLM text encoders capture logic better, though it scales poorly to retrieval.

Limitations

• Logic-aware training data is built only from MSCOCO daily images, and medical performance remains limited for long medical text.
• Negative captions depend on multiple proprietary and open LLMs with possible bias, and positive captions rely on spaCy/regex filtering.
• Evaluation of MLLMs is limited to LLaVA on MCQ; the paper states that the fine-grained objectives trade off general alignment when used alone.

Experimental setup

Datasets: MSCOCO CC12M MSRVTT DADA-2000 Open-i Flickr30K

Benchmarks: LogicBench COCO Flickr30K

Metrics: MCQ accuracy Recall@1 Recall@5

Baselines: NegCLIP NegFull LaCLIP LaionCLIP DatacompCLIP TripletCLIP MetaCLIP ILCLIP ConCLIP OpenAI CLIP-B OpenAI CLIP-L

Builds on: CLIP NegCLIP NegBench

Closest papers by meaning

• JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images (NeurIPS 2024 · similarity 0.90)
• Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.90)
• UniBench: Visual Reasoning Requires Rethinking Vision-Language Beyond Scaling (NeurIPS 2024 · similarity 0.90)
• Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · similarity 0.89)
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.89)
• Synthesizing Visual Concepts as Vision-Language Programs (CVPR 2026 · similarity 0.89)