Research map / Adversarial robustness and security / Attacks on text and graphs
Eyes-on-Me: Scalable RAG Poisoning through Transferable Attention-Steering Attractors
Yen-Shan Chen, Sian-Yao Huang, Cheng-Lin Yang, Yun-Nung Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use random token initialization for the attractor instead of natural-language initialization (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 30.15 -> 42.27 | Improved |
| use random initialization instead of structured tags (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 31.06 -> 42.27 | Improved |
| use random initialization instead of natural-language init (ablation) | Eyes-on-Me + LLM-Gen · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 44.66 -> 71.78 | Improved |
| use a simple forced-sentence instruction rather than information gathering (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 19.41 -> 36.66 | Improved |
| use a simple forced-sentence instruction rather than malicious URL insertion (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 3.98 -> 36.66 | Improved |
| set head-selection correlation threshold at 0.85 rather than 0.9 (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 37.86 -> 44.56 | Improved |
| lower head-selection threshold from 0.85 to 0.8 (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 44.56 -> 16.50 | Hurt |
| steer negatively correlated heads (<-0.85) instead of positive heads (>0.85) (ablation) | Eyes-on-Me · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 44.56 -> 4.72 | Hurt |
| add the Eyes-on-Me attractor to LLM-Gen payloads | LLM-Gen · 18 RAG settings | E2E-ASR: 26.4 -> 57.8 | Improved |
| add LLM-generated retrieval payload to Eyes-on-Me attractor | Eyes-on-Me · Qwen3-Emb-0.6B retriever avg | E2E-ASR: 29.37 -> 64.14 | Improved |
| upgrade PPL filter model from GPT-2 to Qwen2.5-3B-Instruct (ablation) | Eyes-on-Me + LLM-Gen · MS MARCO (BCE, Qwen2.5-0.5B) | E2E-ASR: 76.92 -> 65.80 | Hurt |
| upgrade PPL filter model to Qwen2.5-3B-Instruct (ablation) | Eyes-on-Me + LLM-Gen · MS MARCO (BCE, Qwen2.5-0.5B) | ASR under PPL defense: 72.2 -> 75.2 | Improved |
Closest papers by meaning
- On the Vulnerability of Applying Retrieval-Augmented Generation within Knowledge-Intensive Application Domains (ICML 2025 · similarity 0.85)
- PoisonedEye: Knowledge Poisoning Attack on Retrieval-Augmented Generation based Large Vision-Language Models (ICML 2025 · similarity 0.85)
- Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems (ICLR 2025 · similarity 0.85)
- Shadowcast: Stealthy Data Poisoning Attacks Against Vision-Language Models (NeurIPS 2024 · similarity 0.84)
- Backdoor Attacks in Token Selection of Attention Mechanism (ICML 2025 · similarity 0.84)
- Backdoor Cleaning without External Guidance in MLLM Fine-tuning (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
