Research map / Adversarial robustness and security / Attacks on text and graphs
Chain-of-Thought Driven Adversarial Scenario Extrapolation for Robust Language Models
Md Rafi Ur Rashid, Vishnu Asutosh Dasu, Ye Wang, Gang Tan, Shagufta Mehnaz
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ASE three-step scenario reasoning | GPT-4o · JailBreakV-28k | unsafe response rate: 6.25% -> 0.68% | Improved |
| add ASE | Llama3.3-70B · JailBreakV-28k | unsafe response rate: 62.43% -> 3.15% | Improved |
| add ASE | GPT-4o · JailBreakV-28k | outright rejection: 88.27% -> 10.89% | Improved |
| add ASE | GPT-4o · TruthfulQA (adversarial) | correctness: 74.37% -> 92.45% | Improved |
| add ASE | GPT-4o · RealToxicityPrompts | toxicity score: 35.2 -> 5.5 | Improved |
| add ASE | GPT-4o · HolisticBias | average bias score: 27.5 -> 3.3 | Improved |
| add ASE | GPT-4o · MMLU | accuracy: 78.18% -> 82.04% | Improved |
| add ASE | GPT-4o · CNN/DailyMail | ROUGE-L: 25.67 -> 25.28 | No clear effect |
| use Intention Analysis instead of ASE | GPT-4o · JailBreakV-28k | outright rejection: 97.44% vs ASE 10.89% | Hurt |
| add Paraphrase defense | GPT-4o · JailBreakV-28k | unsafe response rate: 6.25% -> 7.43% | Hurt |
| add Paraphrase defense | GPT-4o · TruthfulQA (adversarial) | correctness: 74.37% -> 67.73% | Hurt |
| apply ASE to a small model | Mistral-7B · JailBreakV-28k | unsafe response rate: 61.79% -> 9.87% | Mixed |
Closest papers by meaning
- Adversarial Reasoning at Jailbreaking Time (ICML 2025 · similarity 0.87)
- Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (NeurIPS 2026 · similarity 0.87)
- Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.86)
- AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models (ICLR 2026 · similarity 0.86)
- Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-Box Neural Ranking Models (AAAI 2025 · similarity 0.86)
- Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts? (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
