Research map / Adversarial robustness and security / Attacks on text and graphs
Dyslexify: A Mechanistic Defense Against Typographic Attacks in CLIP
Lorenz Hufe, Constantin Venhoff, Erblina Purelku, Maximilian Dreyer, Sebastian Lapuschkin, Wojciech Samek
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add typographic circuit ablation (Dyslexify) (ablation) | OpenCLIP ViT-G · ImageNet-100-Typo | accuracy: 46.70 -> 68.76 | Improved |
| add typographic circuit ablation (Dyslexify) (ablation) | OpenCLIP ViT-B · ImageNet-100-Typo | accuracy: 46.94 -> 66.84 | Improved |
| add typographic circuit ablation (Dyslexify) | OpenCLIP ViT-B · Disentangling | accuracy: +31.11 | Improved |
| add typographic circuit ablation (Dyslexify) | OpenCLIP ViT-bigG · Aircraft-T | accuracy: +16.29 | Improved |
| add typographic circuit ablation (Dyslexify) | OpenCLIP ViT-L · Aircraft (non-typographic) | accuracy: -1.74 | Hurt |
| add typographic circuit ablation (Dyslexify) | OpenCLIP ViT-L · Food-101 (non-typographic) | accuracy: -1.17 | Hurt |
| add typographic circuit ablation (Dyslexify) | OpenCLIP ViT-B · IIIT5K OCR small lexicon | accuracy: -30.50 | Hurt |
| add typographic circuit ablation (Dyslexify) | WhyLesionCLIP · Melanoma-T | accuracy: 62.10 -> 81.40 | Improved |
| add typographic circuit ablation (Dyslexify) | WhyLesionCLIP · HAM10k | accuracy: 35.66 -> 40.20 | Improved |
| replace learned Defense-Prefix with Dyslexify | OpenCLIP ViT-L · Disentangling | accuracy: 82.78 -> 60.56 | Hurt |
| replace learned Defense-Prefix with Dyslexify | OpenCLIP ViT-L · RTA-100 | accuracy: 62.20 -> 71.00 | Improved |
| add OCR text-blur preprocessing instead of Dyslexify | OpenCLIP ViT-G · ImageNet-100-Typo | accuracy: 68.76 -> 79.56 | Improved |
Closest papers by meaning
- ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training (NeurIPS 2025 · similarity 0.87)
- Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models (CVPR 2025 · similarity 0.86)
- Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks (ICML 2024 · similarity 0.86)
- Distraction is All You Need for Multimodal Large Language Model Jailbreaking (CVPR 2025 · similarity 0.85)
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts (AAAI 2025 · similarity 0.85)
- When the Prompt Becomes Visual: Vision-Centric Jailbreak Attacks for Large Image Editing Models (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
