Research map / Adversarial robustness and security / Attacks on text and graphs
Read or Ignore? A Unified Benchmark for Typographic-Attack Robustness and Text Recognition in Vision-Language Models
Futa Waseda, Shojiro Yamabe, Daiki Shiono, Kento Sasaki, Tsubasa Takahashi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add text-attack data to training (RIO-RT instead of IT-RT) (ablation) | LLaVA-1.5-7B · Text-VQA attack avg | VQA accuracy: 23.3 -> 52.8 | Improved |
| add text-attack data to training (RIO-RT instead of IT-RT) (ablation) | LLaVA-1.5-13B · Text-VQA attack avg | VQA accuracy: 12.3 -> 50.5 | Improved |
| train on object-attack data only (IT-RT) | LLaVA-1.5-7B · Text-VQA attack avg | VQA accuracy: 45.1 -> 23.3 | Hurt |
| train on object-attack data only (IT-RT) | Qwen3-VL-8B · Text-VQA attack avg | VQA accuracy: 78.5 -> 60.8 | Hurt |
| train on object-attack data only (IT-RT) | LLaVA-1.5-7B · Obj-VQA MC attack avg | accuracy: 71.6 -> 98.4 | Improved |
| balanced read-or-ignore training (RIO-RT) | Qwen2.5-VL-7B · Obj-VQA MC attack avg | accuracy: 65.9 -> 98.6 | Improved |
| balanced read-or-ignore training (RIO-RT) | Llama-3.2-Vision-11B · Text-VQA attack avg | VQA accuracy: 58.2 -> 80.8 | Improved |
| balanced read-or-ignore training (RIO-RT) | SmolVLM-2B · Text-VQA attack avg | VQA accuracy: 61.7 -> 62.9 | No clear effect |
| apply CoT defense prompting | Qwen2.5-VL-7B · Obj-VQA MC attack avg | accuracy: 65.9 -> 77.2 | Improved |
| apply CoT defense prompting | Qwen2.5-VL-7B · Text-VQA attack avg | VQA accuracy: 77.7 -> 68.0 | Hurt |
| apply CoT defense prompting | Qwen3-VL-8B · Obj-VQA MC attack avg | accuracy: 73.2 -> 67.0 | Hurt |
| add language-model LoRA to vision-only tuning (V to V+L) (ablation) | Qwen2.5-VL-7B · Obj-VQA MC attack | accuracy: 62.3 -> 98.5 | Improved |
Closest papers by meaning
- MVI-Bench: A Comprehensive Benchmark for Evaluating Robustness to Misleading Visual Inputs in LVLMs (ICML 2026 · similarity 0.87)
- SceneTAP: Scene-Coherent Typographic Adversarial Planner against Vision-Language Models in Real-World Environments (CVPR 2025 · similarity 0.87)
- FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts (AAAI 2025 · similarity 0.86)
- OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning (NeurIPS 2025 · similarity 0.86)
- ORIC: Benchmarking Object Recognition under Contextual Incongruity in Large Vision-Language Models (CVPR 2026 · similarity 0.86)
- ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
