Research map / Large language models / Retrieval and knowledge
J&H: Evaluating the Robustness of Large Language Models Under Knowledge-Injection Attacks in Legal Domain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes J&H, a syllogism-based legal knowledge-injection attack benchmark that perturbs major premise, minor premise, and conclusion of criminal judgment tasks, showing LLMs are fragile and prompt-level defenses only partially help.
Problem
Whether LLMs make legal judgments through domain knowledge and deductive reasoning, or are misled by surface words, patterns, and irrelevant logic chains when facing knowledge-level perturbations.
Approach
Attacks legal reasoning at each syllogism level (major premise via inserted wrong statutes and similar-crime mentions; minor premise via legal synonym substitution, element insertion, and narration; conclusion via expert-opinion and prior-behavior insertions), measuring accuracy drop (PDR) on multiple-choice crime prediction, and tests RAG, CoT, and few-shot as mitigations.
Key results
- Across models, attacks reduce accuracy with many PDRs above 30%; e.g., LLaMA3 drops 36.67% under provisional element attack and 44.18% under judge expert-opinion attack on LEVEN.
- Conclusion-level attacks are the most effective category; Farui (legal LLM) is the most robust but still drops up to 34.63% under judge opinion attack.
- RAG, CoT, and few-shot reduce PDR in some settings but do not fully solve the vulnerability; CoT can worsen robustness for Baichuan2 and LLaMA3 in some attacks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Insert legal elements from a similar crime into the case facts | LLMs evaluated by J&H · LEVEN | accuracy: 0.777 -> 0.534; 0.734 -> 0.644; 0.671 -> 0.596; 0.679 -> 0.430; 0.849 -> 0.746 | Hurt |
| Insert an opinion that a judge favors a similar crime | LLMs evaluated by J&H · LEVEN | accuracy: 0.777 -> 0.627; 0.734 -> 0.52; 0.671 -> 0.478; 0.679 -> 0.379; 0.849 -> 0.555 | Hurt |
| Use a domain-specific legal model rather than general-domain models | LEVEN legal judgment prediction · LEVEN | accuracy (original): Farui 0.849; general models 0.777, 0.734, 0.671, 0.679 | Helped |
| Add retrieved legal provisions to the prompt | Baichuan2, ChatGLM3, LLaMA3, and Farui · LEVEN | PDR, provisional element attack: 31.27% -> 15.83%; 12.26% -> 5.40%; 36.67% -> 12.59%; 12.13% -> 11.49% | Helped |
| Ask for step-by-step reasoning using the four criminal-law elements | Baichuan2, ChatGLM3, LLaMA3, and Farui · LEVEN | PDR, provisional element attack: 31.27% -> 33.42%; 12.26% -> 10.71%; 36.67% -> 33.72%; 12.13% -> 15.14% | Mixed |
| Add two typical crime and similar-crime examples | Baichuan2, ChatGLM3, LLaMA3, and Farui · LEVEN | PDR, provisional element attack: 31.27% -> 3.17%; 12.26% -> 6.22%; 36.67% -> 33.06%; 12.13% -> 27.29% | Mixed |
| Add legal-element attack rather than factual-element attack | LLMs evaluated by J&H · LEVEN | PDR: Baichuan2 12.36% -> 31.27%; ChatGLM3 5.18% -> 12.26%; GPT3.5 2.98% -> 11.18%; LLaMA3 17.53% -> 36.67%; Farui 4.95% -> 12 | Helped |
Surprising findings
- Chain-of-thought prompting may make robustness worse, suggesting models do not grasp the four-element criminal law reasoning.
- Few-shot prompting failed to improve robustness for several models and in some cases increased PDR for Farui under provisional element attack.
Limitations
- Evaluation limited to multiple-choice crime prediction on Chinese criminal case data (LEVEN, CAIL2018).
- Only a small set of general and legal LLMs evaluated; LLaMA3 described as fine-tuned in Chinese without clear details.
- Mitigations are prompt-level only; no training-based defense evaluated.
Experimental setup
Datasets: LEVEN CAIL2018
Benchmarks: LEVEN CAIL2018
Metrics: accuracy performance drop ratio (PDR)
Baselines: GPT-3.5-turbo Baichuan2-7B-chat ChatGLM3-6B LLaMA3 Farui
Builds on: PromptBench retrieval-augmented generation chain-of-thought LEVEN CAIL2018
Closest papers by meaning
- LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models (NeurIPS 2024 · similarity 0.87)
- CodeCrash: Exposing LLM Fragility to Misleading Natural Language in Code Reasoning (NeurIPS 2025 · similarity 0.86)
- SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain (NeurIPS 2024 · similarity 0.86)
- Rethinking the Development of Large Language Models from the Causal Perspective: A Legal Text Prediction Case Study (AAAI 2024 · similarity 0.85)
- Argumentative Large Language Models for Explainable and Contestable Claim Verification (AAAI 2025 · similarity 0.85)
- CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.