Research map / Vision-language and multimodal / CLIP and zero-shot
Prompt Yourself: Awakening Textual Semantics in 1D Visual Tokenizers
Hualiang Wang, Siming Fu, Weinan Jia, Yuning Lu, Mu Liu, Jidong Jiang, Xiaomeng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-prompted alignment (SPA) (ablation) | VLTok-B-L · ImageNet-1K val | rFID: 0.90 -> 0.72 | Improved |
| add self-prompted alignment (SPA) (ablation) | VLTok-B-L · ImageNet-1K val (MaskGIT) | gFID: 2.08 -> 1.70 | Improved |
| add self-prompted alignment (SPA) (ablation) | VLTok-B-L · ImageNet-1K val | linear probing accuracy: 66.2 -> 69.9 | Improved |
| use SigLIP2 text encoder instead of CLIP-B-16 (ablation) | VLTok-B-L · ImageNet-1K val | rFID: 0.81 -> 0.72 | Improved |
| use SigLIP2 text encoder instead of Qwen2.5-VL-7B (ablation) | VLTok-B-L · ImageNet-1K val | gFID: 2.01 -> 1.70 | Improved |
| use SigLIP2 text encoder instead of T5-XL (ablation) | VLTok-B-L · ImageNet-1K val | rFID: 0.75 -> 0.72 | Improved |
| use Qwen2.5-VL-7B captions instead of LLaVA-Next-3B (ablation) | VLTok-B-L · ImageNet-1K val | rFID: 0.82 -> 0.72 | Improved |
| use Qwen2.5-VL-7B captions instead of InternVL3-8B (ablation) | VLTok-B-L · ImageNet-1K val | gFID: 1.77 -> 1.70 | No clear effect |
| replace contrastive loss with SPA (ablation) | VLTok-B-L (128 tokens) · ImageNet-1K val | rFID: 1.26 -> 1.01 | Improved |
| add contrastive loss to VLTok without alignment (ablation) | VLTok-B-L (128 tokens) · ImageNet-1K val | rFID: 1.42 -> 1.26 | Improved |
| add SPA (out-of-domain) (ablation) | VLTok-B-L · COCO | rFID: 2.28 -> 1.29 | Improved |
| add SPA (out-of-domain) (ablation) | VLTok-B-L · AFHQ-wild | rFID: 14.29 -> 10.28 | Improved |
Closest papers by meaning
- TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generation (CVPR 2025 · similarity 0.91)
- UniTok: a Unified Tokenizer for Visual Generation and Understanding (NeurIPS 2025 · similarity 0.91)
- A More Word-like Image Tokenization for MLLMs (CVPR 2026 · similarity 0.90)
- From Pixels to Tokens: Byte-Pair Encoding on Quantized Visual Modalities (ICLR 2025 · similarity 0.90)
- Image Understanding Makes for A Good Tokenizer for Image Generation (NeurIPS 2024 · similarity 0.90)
- Vision as a Dialect: Unifying Visual Understanding and Generation via Text-Aligned Representations (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
