Research map / Adversarial robustness and security / Attacks on text and graphs
SECA: Semantically Equivalent and Coherent Attacks for Eliciting LLM Hallucinations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SECA, a constraint-preserving zeroth-order search that uses LLM proposer and feasibility checker to find semantically equivalent, coherent prompt rephrasings that elicit incorrect MMLU answers and hallucinated explanations in LLMs.
Problem
Existing adversarial hallucination elicitation for LLMs produces unrealistic prompts (gibberish tokens or meaning-altering rephrasings), so it gives limited insight into real-world hallucinations; the paper seeks realistic prompts that preserve meaning and stay coherent.
Approach
Formulates hallucination elicitation as constrained optimization with semantic equivalence and coherence constraints; SECA iteratively proposes rephrasings with an LLM, filters them with an LLM feasibility checker for semantic equivalence, and keeps the top-N candidates maximizing log-probability of a target incorrect answer token. Coherence is enforced by using LLM-generated prompts and checked v
Key results
- SECA raises ASR@30 on Llama-3-3B from 48.20% (raw) to 80.29% and on Qwen-2.5-7B from 10.19% to 36.86%, while GCG reaches only 0.57-9.86%.
- SECA keeps SEE at 0.00 for all tested models, whereas GCG has SEE around 0.96-0.98.
- The feasibility checker has perfect recall (1.000) and F1 of 0.873 / 0.808 against two human annotators; the hallucination evaluator agrees with humans with accuracy 0.880-0.940.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SECA instead of raw prompts | Llama-3-3B · MMLU filtered subset | ASR@30: 48.20 -> 80.29 | Helped |
| Use SECA instead of GCG | Llama-3-3B · MMLU filtered subset | ASR@30: 6.26 -> 80.29 | Helped |
| Use SECA instead of raw prompts | Llama-3-8B · MMLU filtered subset | ASR@30: 63.52 -> 81.24 | Helped |
| Use SECA instead of GCG | Llama-3-8B · MMLU filtered subset | ASR@30: 9.86 -> 81.24 | Helped |
| Use SECA instead of raw prompts | Qwen-2.5-7B · MMLU filtered subset | ASR@30: 10.19 -> 36.86 | Helped |
| Use SECA instead of GCG | Qwen-2.5-7B · MMLU filtered subset | ASR@30: 0.57 -> 36.86 | Helped |
| Use SECA instead of raw prompts | Llama-3-3B · MMLU filtered subset | SCE: 1.08 -> 0.60 | Helped |
| Use SECA instead of raw prompts | Llama-3-8B · MMLU filtered subset | SCE: 1.08 -> 0.33 | Helped |
| Use SECA instead of raw prompts | Qwen-2.5-7B · MMLU filtered subset | SCE: 1.08 -> 1.06 | No clear effect |
| Use GCG instead of SECA | Llama-3-3B · MMLU filtered subset | SCE: 0.60 -> 1255.04 | Hurt |
| Use GCG instead of SECA | Llama-3-8B · MMLU filtered subset | SCE: 0.33 -> 307.68 | Hurt |
| Use GCG instead of SECA | Qwen-2.5-7B · MMLU filtered subset | SCE: 1.06 -> 1036.62 | Hurt |
Surprising findings
- GCG's gibberish suffix lowers the target-token objective so much that its ASR@30 falls below that of raw unmodified prompts.
- Successful SECA prompts tend to be more lexically diverse and longer than the original prompts.
Limitations
- Only targeted, open-ended multiple-choice setting evaluated on MMLU; extension to free-form generation left to future work.
- Feasibility checker has false positives (lower precision), and GPT-4.1-Nano proposer may hallucinate.
- Zeroth-order search is costly (8-16 GPU hours per model on 100 samples).
Experimental setup
Datasets: MMLU
Benchmarks: MMLU
Metrics: ASR@30 semantic equivalence error (SEE) semantic coherence error (SCE) type-token ratio
Baselines: Raw MMLU prompts GCG
Builds on: GCG
Closest papers by meaning
- Look Closer! An Adversarial Parametric Editing Framework for Hallucination Mitigation in VLMs (AAAI 2026 · similarity 0.91)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.90)
- Poison as Cure: Visual Noise for Mitigating Object Hallucinations in LVMs (NeurIPS 2025 · similarity 0.90)
- Analyzing and Mitigating Object Hallucination: A Training Bias Perspective (AAAI 2026 · similarity 0.90)
- Intervene-All-Paths: Unified Mitigation of LVLM Hallucinations across Alignment Formats (NeurIPS 2025 · similarity 0.90)
- Combating Multimodal LLM Hallucination via Bottom-Up Holistic Reasoning (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.