Research map / Adversarial robustness and security / Attacks on text and graphs
MathAttack: Attacking Large Language Models towards Math Solving Ability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MathAttack, a black-box attack that freezes logical entities (roles, numbers, scenarios) in math word problems and applies a BERT-based word-level attack, and releases the RobustMath adversarial dataset to test LLM math robustness.
Problem
LLMs' robustness in math word problem solving has been little studied, and generic text adversarial attacks tend to alter the mathematical logic of problems, invalidating the ground-truth answer.
Approach
Identify logical entities (role, number, scenario) with spaCy NER, freeze them, and attack the remaining words with a BERT-Attack-style word-level attacker that ranks words by masking-induced drop in correct-answer probability; adversarial samples are manually checked to preserve math logic.
Key results
- ChatGPT zero-shot ASR is 41.15% on GSM8K and 39.19% on MultiArith, dropping to 19.93% and 21.09% with few-shot prompts.
- Adversarial samples from ChatGPT transfer to Flan-T5-large and Flan-T5-xl with 94.44% and 89.47% TSR, while smaller models transfer to ChatGPT at only 6.25% and 6.67%.
- Using adversarial examples as few-shot prompts lowers ChatGPT ASR to 14.07% on GSM8K and 16.33% on MultiArith while keeping clean accuracy around 88-98%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use few-shot prompts instead of zero-shot prompts | ChatGPT under MathAttack · GSM8K | ASR: 41.15 -> 19.93 | Hurt |
| Use few-shot prompts instead of zero-shot prompts | ChatGLM2 under MathAttack · GSM8K | ASR: 56.29 -> 64.82 | Mixed |
| Use MathAttack samples with correct labels as few-shot examples | Large language models · GSM8K | ASR: 14.07 | Hurt |
| Use MathAttack samples with correct labels as few-shot examples | Large language models · MultiAirth | ASR: 16.33 | Hurt |
| Fine-tune Flan-T5-xl on 200k MWP data | Flan-T5-xl · RobustMath, zero-shot | accuracy: 6.00 -> 12.33 | Helped |
| Generate adversarial samples from a larger model and transfer them to a smaller model | ChatGPT adversarial samples · Transfer to Flan-T5-large | TSR: 94.44% | Helped |
| Generate adversarial samples with few-shot prompts and transfer them to zero-shot | ChatGPT · Transfer from few-shot to zero-shot | TSR: 45.24% | Helped |
| Increase the number of reasoning steps from 2 to 3 | MathAttack · MWPs | ASR | Helped |
| Increase problem length | MathAttack · MWPs | ASR | Helped |
| Increase the number of numbers in a problem | MathAttack · MWPs | ASR | Helped |
Surprising findings
- Smaller models' adversarial samples fail to transfer to larger models, while larger models' samples transfer well to smaller ones.
- Few-shot prompts with adversarial samples (with correct labels) substantially improve robustness against attack.
Limitations
- Attack relies on manual verification to ensure mathematical logic is preserved.
- Only four victim models and simple MWP subsets (307 GSM8K, 150 MultiArith) are evaluated.
- Expensive generation process requiring many queries per sample.
Experimental setup
Datasets: GSM8K MultiArith
Benchmarks: GSM8K MultiArith RobustMath
Metrics: Clean Acc Attack Acc Attack Success Rate Similarity (Universal Sentence Encoder) Transfer Success Rate
Baselines: Prompt-Attack (ICL attack) PromptBench SVAMP
Builds on: BERT-Attack chain-of-thought TextAttack
Closest papers by meaning
- An LLM can Fool Itself: A Prompt-Based Adversarial Attack (ICLR 2024 · similarity 0.88)
- Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.87)
- A Careful Examination of Large Language Model Performance on Grade School Arithmetic (NeurIPS 2024 · similarity 0.87)
- Exploiting LLM Quantization (NeurIPS 2024 · similarity 0.87)
- MathArena: Evaluating LLMs on Uncontaminated Math Competitions (NeurIPS 2025 · similarity 0.87)
- Context-Aware Hierarchical Learning: A Two-Step Paradigm towards Safer LLMs (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.