Research map / Adversarial robustness and security / Backdoor and poisoning

Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models

NeurIPS 2026 · Adversarial robustness and security › Backdoor and poisoning · approach: Backdoors in large multimodal models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that unified autoregressive models (Liquid, JanusPro) and autoregressive text-to-image generators can be backdoored via a trigger word or homoglyph that induces poisoned image tokens which then elicit poisoned text, via black-box data poisoning and white-box teacher-guided logit distillation.

Problem

Backdoor vulnerabilities of autoregressive image generation and unified autoregressive models that generate text and images jointly remain unexplored; existing backdoor work targets diffusion models or unimodal outputs.

Approach

A transitive self-poisoning chain: trigger text -> poisoned image tokens -> poisoned text, enabled by overlapping poisoned pairs (black-box) or hook (logit KL) plus link (cross-entropy) losses (white-box), with bidirectional training on overlapping pairs proposed as a defense.

Key results

• Black-box poisoning with 1% injection rate on JanusPro yields up to 91.0% visual ASR for the proud->rainbow flag scenario and 90.3% joint ASR_U.
• White-box Liquid achieves 81.10% visual ASR (human-eval 90.20%) for the cool/omicron-smoking trigger in T2I poisoning.
• Training under random T2I/I2T flipping of overlapping pairs reduces Liquid's ASR_U from 57.20% to 2.40% in the freedom->anarchy scenario, while preserving FID and POPE.

Surprising findings

• Homoglyph triggers are slightly stronger than common-word triggers on average, but the common-word triggers are harder to detect by existing prompt-level scanners.
• Vision-triggered activation transfers: a poisoned anarchy image alone induced harmful text in 84% of cases with JanusPro.

Limitations

• Only two unified models (Liquid, JanusPro) fully evaluated; Emu3 only T2I stage and Chameleon/Tuna excluded.
• White-box attack depends on teacher's ability to generate the target concept.
• Defense (bidirectional flipping) evaluated only for Liquid black-box setting.

Experimental setup

Datasets: MJHQ-30K DiffusionDB

Benchmarks: MJHQ-30K POPE DiffusionDB

Metrics: ASR_V ASR_V-HE ASR_T ASR_U FID POPE clean rate

Baselines: clean data fine-tuning T2T link Unaligned I2T link

Builds on: LoRA Janus Liquid Emu3 KL distillation

Closest papers by meaning

• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.88)
• BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models (CVPR 2025 · similarity 0.88)
• Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models (AAAI 2024 · similarity 0.86)
• Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift (CVPR 2025 · similarity 0.86)
• Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models (AAAI 2025 · similarity 0.86)
• Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.86)