AI Paper Atlas

Research map / Large language models / Agents, code and math

Can LLMs Reason Soundly in Law? Auditing Inference Patterns for Legal Judgment

Lu Chen, Yuxuan Huang, Yixing Li, Dongrui Liu, Qihan Ren, ShuaiZhao, Kun Kuang, Zilong Zheng, Quanshi Zhang

ICLR 2026 · Large language models › Agents, code and math · approach: Mathematical and structural reasoning benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
audit legal LLM judgments by interaction patterns rather than output correctnessQwen2.5-14B, DeepSeek-R1-Distill-Qwen-14B, SaulLM-7B, BAI-Law-13B · ECtHR, LegalBench, CAIL2018, LeCaRD, LEVENs_reliableHurt
judge defendant using text describing another defendant's actionsSaulLM-7B-Instruct · CAIL2018 Andy assault cases_reliable: 41.5% reliableHurt
law-specific fine-tuning (BAI-Law-13B vs SaulLM-7B) on the same caselegal LLMs · CAIL2018 Andy cases_reliable: 41.5% -> 44.5%No clear effect
replace victim occupation 'a judge' with 'a volunteer' or 'a programmer' (ablation)SaulLM-7B-Instruct · Robbery casejudgment output / reliable interaction effect: 0.22 -> 0.06; Robbery -> Not mentionedHurt
use law-related occupations (lawyer, policeman) instead of judge (ablation)SaulLM-7B-Instruct · Robbery casejudgment outputNo clear effect
inspect interaction orders for chain-of-thought-like reasoningevaluated legal LLMs · LegalBench, CAIL2018, LeCaRD, LEVENinteraction order distributionHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy