Research map / Adversarial robustness and security / Attacks on text and graphs
Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adversarial Paraphrasing, a training-free attack that guides an instruction-tuned LLM paraphraser (LLaMA-3-8B-Instruct) with an AI-text detector's scores at each decoding step to humanize AI text and evade diverse detectors.
Problem
AI-generated text detectors, including recent paraphrase-robust ones, remain vulnerable to evasion; the paper asks whether a universal, transferable attack can bypass a wide range of neural, watermark and zero-shot detectors.
Approach
At each autoregressive step, the paraphraser LLM proposes top-p/top-k candidate tokens and the token whose continuation receives the lowest AI score from a guidance detector is chosen (depth-one detector-guided search), requiring no training and no gradients.
Key results
- Adversarial paraphrasing guided by OpenAI-RoBERTa-Large reduces T@1%F by 98.96% on Fast-DetectGPT versus simple paraphrasing's 15.03% increase, and averages 87.88% T@1%F reduction across eight detectors.
- On RADAR, adversarial paraphrasing guided by RoBERTa-Large reduces T@1%F by 64.49% where simple paraphrasing increased it by 8.57%.
- GPT-4o rated 87% of adversarial paraphrases as 4 or 5 out of 5, with quality differences from simple paraphrasing not statistically significant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide paraphrasing with OpenAI-RoBERTa-Large rather than use simple paraphrasing | AI-generated text detection on MAGE and watermarked MAGE · RADAR and Fast-DetectGPT | T@1%F: reduces T@1%F by 64.49% on RADAR and 98.96% on Fast-DetectGPT | Helped |
| Use adversarial paraphrasing instead of simple paraphrasing | RADAR detection · MAGE | T@1%F: 0.140 -> 0.044 | Helped |
| Use adversarial paraphrasing instead of simple paraphrasing | Fast-DetectGPT detection · MAGE | T@1%F: 0.326 -> 0.003 | Helped |
| Use simple paraphrasing instead of no attack | RADAR detection · MAGE | T@1%F: 0.124 -> 0.140 | Hurt |
| Use simple paraphrasing instead of no attack | Fast-DetectGPT detection · MAGE | T@1%F: 0.323 -> 0.326 | Hurt |
| Set top-p to 0.992 instead of 0.99 (ablation) | AdvPara with OpenAI-RoBERTa-Large guidance · All 8 deployed detectors | mean T@1%F: 0.0690 -> 0.0565 | Helped |
| Set top-p to 0.5 instead of 0.99 (ablation) | AdvPara with OpenAI-RoBERTa-Large guidance · All 8 deployed detectors | mean T@1%F: 0.0690 -> 0.2985 | Hurt |
| Vary top-k from 50 to 10, 25, 75, or 100 (ablation) | AdvPara with OpenAI-RoBERTa-Large guidance · All 8 deployed detectors | mean T@1%F: 0.0690; 0.0757; 0.0658; 0.0698; 0.0698 | No clear effect |
| Apply detector guidance once every 2–5 decoding steps instead of every step (ablation) | AdvPara with OpenAI-RoBERTa-Large guidance · All 8 deployed detectors | Mean T@1%FPR: 0.0740 -> 0.1333 (N=2) -> 0.1690 (N=3) -> 0.1820 (N=4) -> 0.2115 (N=5) | Hurt |
| Use adversarial paraphrasing rather than simple paraphrasing | GPT-4o text quality ratings · MAGE, KGW watermarked MAGE, and Unigram watermarked MAGE | quality rating | No clear effect |
| Use adversarial paraphrasing rather than simple paraphrasing | semantic preservation · MAGE | SBERT cosine similarity: 0.8601 ± 0.0880 -> 0.8082 ± 0.1006 | Hurt |
Surprising findings
- Simple paraphrasing increases T@1%F on RADAR and Fast-DetectGPT rather than reducing it.
- About 4.15% of samples backfired, with the paraphrase becoming more likely to be flagged as AI in the transfer setting.
Limitations
- Attack sometimes fails (about 7.4% of samples) or backfires (about 4.15%) in the transfer setting.
- Quality evaluated mainly via GPT-4o ratings and perplexity; no human evaluation.
- Adds detector inference at every decoding step, increasing latency relative to simple paraphrasing.
Experimental setup
Datasets: MAGE KGW watermarked MAGE texts Unigram watermarked MAGE texts
Benchmarks: MAGE
Metrics: AUC T@1%F GPT-4o quality rating perplexity SBERT cosine similarity win rate
Baselines: Simple Paraphrase Recursive Paraphrase Watermark Stealing RADAR Fast-DetectGPT
Builds on: DIPPER BEAST Fast-DetectGPT RADAR InstructCTG
Closest papers by meaning
- DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios (NeurIPS 2024 · similarity 0.88)
- OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples (AAAI 2024 · similarity 0.87)
- ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training (NeurIPS 2025 · similarity 0.86)
- Position: On the Possibilities of AI-Generated Text Detection (ICML 2024 · similarity 0.86)
- IPAD: Inverse Prompt for AI Detection - A Robust and Interpretable LLM-Generated Text Detector (NeurIPS 2025 · similarity 0.86)
- Human Texts Are Outliers: Detecting LLM-generated Texts via Out-of-distribution Detection (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.