AI Paper Atlas

Research map / Vision-language and multimodal / CLIP and zero-shot

TTD: Text-Tag Self-Distillation Enhancing Image-Text Alignment in CLIP to Alleviate Single Tag Bias

Sanghyun Jo, Soohyun Ryu, Sungyub Kim, Eunho Yang, Kyungsu Kim

ECCV 2024 · Vision-language and multimodal › CLIP and zero-shot · approach: Fine-grained improvements to CLIP · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add L_distill (text-tag self-distillation) (ablation)TCL fine-tuning · Pascal VOC / text-levelmIoU: 58.5 -> 61.1Improved
add L_tag only (without L_distill) (ablation)TCL fine-tuning · Text-level segmentationCaptionIoU: 60.4 -> 60.7No clear effect
select tags by pixel-tag scoring instead of NLTK (ablation)TTD pseudo-tagging · Pascal VOCmIoU: 56.5 -> 61.1Improved
select tags by global image-tag scoring instead of pixel-tag scoring (ablation)TTD pseudo-tagging · Pascal VOCmIoU: 55.0 -> 52.5Hurt
add TTD fine-tuning to MaskCLIPMaskCLIP · 6 open-vocab segmentation benchmarks (avg)mIoU: 19.0 -> 27.5Improved
add TTD fine-tuning to TCLTCL · 6 open-vocab segmentation benchmarks (avg)mIoU: 30.4 -> 33.9Improved
use pixel-tag scoring for multi-tag selection instead of global image scoringTCL encoders · CC3M val multi-tag selectionF1: 43.7 -> 78.5Improved
prune training samples to those with above mean+std image-text similarity (ablation)TTD training · Pascal VOCmIoU: 60.5 -> 61.1 with 247 -> 34 hoursImproved
apply TTD versus TagAlign on large-class datasetsTCL + TTD · COCO-Stuff, ADE20kmIoU: COCO-Stuff 23.7 vs 25.3; ADE 17.0 vs 17.3Hurt

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy