Research map / Adversarial robustness and security / Black-box and transfer attacks
Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-Box Neural Ranking Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Attack-in-the-Chain (AttChain), a CoT-style black-box attack that uses LLMs (GPT-3.5, Llama-3-8B) to iteratively perturb low-ranked documents against neural ranking models by selecting anchor documents and assigning perturbation budgets based on ranking gaps.
Problem
Black-box adversarial ranking attacks promoting low-ranked target documents in neural ranking models (NRMs) without surrogate models, while remaining imperceptible and query-efficient.
Approach
Treat higher-ranked anchor documents as nodes in an LLM-driven reasoning chain: filter candidates with a Zipf-based distribution, let the LLM select anchors, assign perturbation word counts proportional to the rank gap to the anchor, generate perturbations, and keep the node giving the best rank improvement for the next round.
Key results
- On MS MARCO against RankLLM (Easy targets), AttChain-GPT reaches 100.0 ASR, 40.5 Boost and 87.5 T10R, versus best baseline IDEM at 98.9, 34.8 and 79.2.
- AttChain-GPT needs about 25 queries to the target NRM on average versus 72-218 for baselines when attacking RankLLM on MS MARCO.
- AttChain-GPT yields lower spamicity detection rates than all baselines (e.g., 6.3% vs 9.1% for PAT at threshold 0.08).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AttChain_GPT instead of the strongest listed baseline | BERT ranking attacks · MS MARCO Easy | Boost: 39.6 -> 44.5 | Helped |
| Use AttChain_GPT instead of the strongest listed baseline | BERT ranking attacks · MS MARCO Hard | Boost: 69.6 -> 91.2 | Helped |
| Use AttChain_GPT instead of the strongest listed baseline | PROP ranking attacks · TREC2019 Hard | Boost: 66.0 -> 88.3 | Helped |
| Use AttChain_GPT instead of the strongest listed baseline | RankLLM ranking attacks · TREC2019 Hard | Boost: 60.9 -> 83.6 | Helped |
| Add LLM-based anchor document selection instead of rotating the top five documents (ablation) | AttChain_GPT · MS MARCO Easy with BERT | Boost: 37.1 -> 44.5 | Helped |
| Add discrepancy-oriented dynamic perturbation instead of using the same degree each round (ablation) | AttChain_GPT · MS MARCO Hard with BERT | Boost: 84.3 -> 91.2 | Helped |
| Use AttChain_GPT instead of PRADA | RankLLM attacks · MS MARCO Mixture | PPL: 102.8 -> 38.3 | Helped |
| Use AttChain_GPT instead of IDEM | RankLLM attacks · MS MARCO Mixture | Imperceptibility: 0.75 -> 0.85 | Helped |
Surprising findings
- Highest-ranked document is not necessarily the best anchor; AttChain-CoT using top-5 anchors performs worse than LLM-selected anchors.
- Perplexity-based detection of AttChain adversarial examples overlaps heavily with original documents.
Limitations
- Relies on closed-source GPT-3.5, which improves effectiveness but incurs substantial cost.
- Only evaluated on two web search benchmarks with three NRMs (BERT, PROP, RankLLM).
- Perplexity and semantic similarity detection are shown to be weak, but other defenses like LLM-generated text detection are not evaluated.
Experimental setup
Datasets: MS MARCO Document Ranking TREC DL 2019
Benchmarks: MS MARCO Document Ranking TREC DL 2019
Metrics: ASR Boost T10R Qrs spamicity detection (OSD) grammar errors PPL human imperceptibility
Baselines: Term spamming (TS) PRADA PAT IDEM
Builds on: chain-of-thought prompting PRADA PAT IDEM
Closest papers by meaning
- BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models (ICLR 2024 · similarity 0.87)
- RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content (ICML 2024 · similarity 0.86)
- Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.86)
- BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.86)
- Adversarial Search Engine Optimization for Large Language Models (ICLR 2025 · similarity 0.86)
- Semantic Representation Attack against Aligned Large Language Models (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.