AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Can the Parts Fool the Test? Counterfactual Pairing Cycles for Relational OOD

Yibo Zhou, Yirong Xiang

NeurIPS 2026 · Vision-language and multimodal › Human and embodied modeling · approach: Multimodal retrieval-augmented modeling · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes counterfactual pairing cycles to build evaluations for relational out-of-distribution errors where the caption's relation is wrong for the image, controlling for shortcuts like caption style and object frequency.

Problem

Relational out-of-distribution errors (captions describing the wrong relation for an image) are hard to evaluate cleanly, because many existing negatives can be solved via superficial shortcuts such as caption style, template, length, or object frequency rather than checking the image-caption pairin

Approach

Uses counterfactual pairing cycles to construct evaluation pairs in which negatives cannot be distinguished from positives by surface cues, so that solving the task requires verifying the relational pairing. Details beyond the abstract are not available.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with cycle contrast loss on exact cycles instead of shortcut-contaminated BCErank-64 bilinear score on frozen CLIP ViT-B/32 · COCO exact cycles (OE stress test)CycleAcc: 0.8817 -> 0.9270Improved
train four-entry BCE on exact cycles instead of cycle contrast loss (ablation)rank-64 bilinear score on frozen CLIP ViT-B/32 · COCO exact cycles (OE stress test)CycleAcc: 0.9270 -> 0.8853Hurt
inject marker prefix shortcut into OE negatives and validate on itOE model · COCO exact cyclesCycleAcc: shortcut val AUROC 0.9998; CycleAcc 0.8817 vs 0.9270Mixed
increase bilinear rank from 1 to 64 (ablation)bilinear probe on frozen CLIP · COCO object-matched cyclesCycleAcc: 0.784 -> 0.959Improved
increase bilinear rank from 1 to 64 (ablation)bilinear probe on frozen CLIP · COCO strict present left/right cyclesCycleAcc: 0.514 -> 0.505No clear effect
increase bilinear rank from 1 to 64 (ablation)bilinear probe on frozen CLIP · COCO strict same-pair relation cyclesCycleAcc: 0.567 -> 0.599Improved
increase bilinear rank from 1 to 64 (ablation)bilinear probe on frozen CLIP · COCO strict size cyclesCycleAcc: 0.687 -> 0.787Improved
increase rank from 8 to 16 in synthetic rank control (ablation)interaction probe · Synthetic rank bottleneck controlAUROC: 0.7658 -> 0.7575No clear effect
use additive image-only, text-only or additive probes under exact cyclesadditive probes · COCO and GQA exact cyclesCycleAcc: 0.500 ± 0.000No clear effect
inject marginal artifact into unblocked synthetic diagnosticmarginal probe · Synthetic controlsAUROC: 0.5005 -> 1.0000Improved
use pair-aware features instead of bag features (ablation)binding representation control · Synthetic binding controlAUROC: 0.5000 -> 0.9919Improved

Surprising findings

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy