Research map / Vision-language and multimodal / CLIP and zero-shot
Left–Right Symmetry Breaking in CLIP-Style Vision-Language Models Trained on Synthetic Spatial-Relation Data
Takaki Yamamoto, Chihiro Noguchi, Toshihiro Tanizawa
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase number of training object labels (label diversity) (ablation) | CLIP-style Transformer on 1D toy data · unseen-pair, seen-pair and single-object generalization | image-to-text retrieval accuracy | Improved |
| increase layout diversity (spatial configurations) (ablation) | CLIP-style Transformer on 1D toy data · generalization accuracy | image-to-text retrieval accuracy | No clear effect |
| add weight decay regularization (ablation) | CLIP-style training · toy generalization | generalization accuracy | Improved |
| zero the token-position (EP) cross term in attention logits (ablation) | vision encoder · unseen-pair generalization | left-right discrimination accuracy: drops to near 0.5 | Hurt |
| ablate position-dependent value term PW_V (ablation) | vision encoder · unseen-pair generalization | accuracy: decreases to 0.5 | Hurt |
| train with both left and right textual captions using 1-layer text encoder (ablation) | CLIP-style model · unseen-pair generalization | top-2 retrieval correctness | Hurt |
| add a second layer to the text encoder with both left and right captions (ablation) | text encoder · unseen-pair generalization | generalization accuracy | Improved |
| extend from 1D to 4x4 2D images | toy model · unseen-pair generalization | accuracy: 98% accuracy | No clear effect |
Closest papers by meaning
- Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence (CVPR 2024 · similarity 0.88)
- Object-centric binding in Contrastive Language-Image Pretraining (NeurIPS 2025 · similarity 0.86)
- Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.85)
- Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas (ICML 2025 · similarity 0.85)
- SpatialCLIP: Learning 3D-aware Image Representations from Spatially Discriminative Language (CVPR 2025 · similarity 0.85)
- SmartCLIP: Modular Vision-language Alignment with Identification Guarantees (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
