Research map / Vision-language and multimodal / Human and embodied modeling
CLAY: Conditional Visual Similarity Modulation in Vision-Language Embedding Space
Sohwi Lim, Lee Hyoseok, Jungjoon Park, Tae-Hyun Oh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CLAY conditional modulation to SigLIP-B | SigLIP-B · Cat species | mAP: 55.9 -> 82.1 | Improved |
| add CLAY conditional modulation to CLIP-B | CLIP-B · Clevr4 Color | mAP: 19.9 -> 67.9 | Improved |
| add CLAY conditional modulation to SigLIP-B | SigLIP-B · CLAY-Object Color | mAP: 14.7 -> 63.3 | Improved |
| add CLAY modulation to SigLIP-B | SigLIP-B · Stanford40 Mood | mAP: 56.4 -> 58.7 | Hurt |
| use CLAY instead of InstructBLIP | SigLIP-B · Clevr4 Shape | mAP: 83.4 -> 88.6 | Mixed |
| extend CLAY to multiple conditions with CLIP-B | CLIP-B · CLAY-Human All conditions | mAP: 31.9 -> 58.0 | Improved |
| add rotation operation to manifold modeling (ablation) | CLAY · Stanford40 | mAP: 57.9 -> 59.8 | Improved |
| add rotation operation to manifold modeling (ablation) | CLAY · CLAY-Object | mAP: 63.7 -> 66.9 | Improved |
| add manifold modeling without rotation (ablation) | CLAY · CLAY-Object | mAP: 65.4 -> 63.7 | Hurt |
| add manifold modeling without rotation (ablation) | CLAY · Stanford40 | mAP: 57.9 -> 57.9 | No clear effect |
| use symmetric formulation (condition database too) for GeneCIS (ablation) | GeneCIS · CLAY-Human | mAP: 51.2 -> 54.8 | Improved |
| use symmetric formulation for Qwen3-VL-Embedding (ablation) | Qwen3-VL · Fine-grained | mAP: 68.6 -> 85.3 | Improved |
Closest papers by meaning
- The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric (NeurIPS 2026 · similarity 0.90)
- Asymmetric Visual Semantic Embedding Framework for Efficient Vision-Language Alignment (AAAI 2025 · similarity 0.88)
- Revisiting the Role of Language Priors in Vision-Language Models (ICML 2024 · similarity 0.88)
- Bridging Vision and Language Spaces with Assignment Prediction (ICLR 2024 · similarity 0.88)
- Same or Not? Enhancing Visual Perception in Vision-Language Models (CVPR 2026 · similarity 0.88)
- VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
