Research map / Adversarial robustness and security / Attacks on text and graphs

DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systems

NeurIPS 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: Adversarial prompts for language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DECEIVE-AFC, an agent-based black-box claim-level adversarial attack framework that rewrites claims with search-disrupting and reasoning-disrupting strategies to flip verdicts of search-enabled LLM fact-checkers, validated on MOCHEG across HiSS, LEMMA, and DEFAME.

Problem

Search-enabled LLM-based fact-checking systems are vulnerable to adversarial claims, but existing claim-level attacks designed for static-corpus DNN pipelines rarely cause system-level verification failures against them.

Approach

A multi-agent closed-loop framework (generator, surrogate victim, evaluator, semantic guard, planner) that iteratively refines claims using three strategy families (search engine misguidance, LLM reasoning disruption, structural complexity escalation) under semantic similarity and NLI validity constraints.

Key results

• On the surrogate GPT-4o-search-preview victim, DECEIVE-AFC cuts accuracy from 90.5% to 57.4% with 36.6% ASR.
• On DEFAME, DECEIVE-AFC reaches 31.4% ASR versus 4.3-8.9% for the FACTEVAL-based baselines; on HiSS and LEMMA it reaches 19.5% and 18.8%.
• Removing the semantic guard lowers ASR from 36.6% to 23.1% even though label flip rate rises slightly (43.3% vs 42.1%).

Surprising findings

• Removing the semantic guard slightly increases label flip rate but substantially decreases valid attack success, since many flips are semantically invalid.
• Existing character-level claim perturbations (LEET, Homoglyph, Character Swap, Phonetic) yield only 4-9% ASR against search-enabled systems.

Limitations

• Surrogate victim (GPT-4o-search-preview) is used during refinement, so transfer to the real target systems is only partially established; ASR on HiSS and LEMMA is below 20%.
• Final-stage validity relies on relaxed thresholds and human validation for some cases.
• Evaluation covers only MOCHEG with NEI samples removed.

Experimental setup

Datasets: MOCHEG

Benchmarks: MOCHEG

Metrics: accuracy precision recall F1 attack success rate ROUGE-1 ROUGE-2 ROUGE-L MAUVE SummaC

Baselines: LEET Homoglyph Character Swap Phonetic

Builds on: FACTEVAL HiSS LEMMA DEFAME

Closest papers by meaning

• Bits Leaked per Query: Information-Theoretic Bounds for Adversarial Attacks on LLMs (NeurIPS 2025 · similarity 0.87)
• Adversarial Reasoning at Jailbreaking Time (ICML 2025 · similarity 0.85)
• An LLM can Fool Itself: A Prompt-Based Adversarial Attack (ICLR 2024 · similarity 0.85)
• How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions (ICLR 2024 · similarity 0.85)
• Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.85)
• Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs (NeurIPS 2026 · similarity 0.84)