Research map / Large language models / Language, safety and interpretability
STEAD: Robust Provably Secure Linguistic Steganography with Diffusion Language Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STEAD, a provably secure and robust linguistic steganography method built on the diffusion language model Dream, embedding messages at parallel-denoised positions with repetition codes, pseudo-random error correction, and neighborhood search extraction.
Problem
Provably secure linguistic steganography based on autoregressive models suffers error propagation, so stegotext tampering (substitution, insertion, deletion, tokenization ambiguity) breaks message extraction.
Approach
Use the parallel, independent unmasking of positions within a single diffusion denoising step as robust embedding positions, embedding repeated message bits via message-driven pseudo-random sampling with a repetition code, and recovering from token errors with pseudo-random error correction and a mu-neighborhood search for alignment shifts.
Key results
- STEAD on Dream with top-p 1.0 reaches 84.08 bit per 10^3 tokens capacity versus 13.81 for PSARS on Qwen2 under the same sampling settings.
- Under mixed token attacks, STEAD keeps extraction success rate of 99.49% on Dream without attack in the migration table, versus 95.50% for Discop on Qwen.
- With a word substitution rate of 0.1 (TextAttack synonym attack), STEAD sustains an extraction correct rate above 80% while non-robust methods are almost ineffective.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Migrate Discop from an autoregressive model to Dream | Discop · IMDb token ambiguity | success rate (%): 97.50 -> 0.00 | Hurt |
| Use STEAD instead of directly migrating Discop to Dream | Dream · IMDb token ambiguity | success rate (%): 0.00 -> 99.49 | Helped |
| Add robust position embedding with error-correction coding (ablation) | STEAD · Token substitution, α=0.2 | message extraction robustness | Helped |
| Add neighborhood search extraction (ablation) | STEAD · Token insertion and deletion, β*=10 and γ*=10 | message extraction robustness | Helped |
| Remove robust position embedding, error-correction coding, and neighborhood search (ablation) | STEAD · Token substitution, insertion, and deletion | message extraction robustness | Hurt |
| Increase neighborhood window from μ=1 to μ=2 (ablation) | STEAD · Strong mixed token attack | correct rate (%): 70.99 -> 88.80 | Helped |
| Increase neighborhood window from μ=2 to μ=4 (ablation) | STEAD · Strong mixed token attack | correct rate (%): 88.80 -> 92.87 | Helped |
| Add a (20, 1) repetition code to Discop (ablation) | Discop on Qwen · Strong mixed token attack | correct rate (%) | No clear effect |
Surprising findings
- Directly migrating ARM-based Discop to a diffusion model drops its success rate to 0.00%, showing naive porting does not give robustness.
- The baseline STEAD without RPE, ECC and NSE performs comparably to ARM-based robustness, so DLM alone does not explain the gains.
Limitations
- Robustness theorem bound 2(alpha+beta+gamma) < min N_s/L is strict; min N_s is typically 3, giving roughly 0.3% tolerable tampering for 512 tokens.
- Embedding capacity is lower under top-p below 1.0 and the repetition code trades capacity for robustness.
- Evaluation uses only 200 IMDb prompts and a single DLM (Dream), with limited comparison to recent robust methods.
Experimental setup
Datasets: IMDb C4
Benchmarks: IMDb prompts C4 prompts
Metrics: embedding capacity entropy encoding and decoding rate perplexity detection error rate P_E correct rate wrong rate lost rate success rate
Baselines: PSARS Discop SparSamp Meteor ARS
Builds on: Discop SparSamp PSARS masked diffusion language models
Closest papers by meaning
- Promising Multi-Granularity Linguistic Steganography by Jointing Syntactic and Lexical Manipulations (AAAI 2025 · similarity 0.87)
- Provably Secure Image Robust Steganography via Cross-modal Error Correction (AAAI 2025 · similarity 0.86)
- SSHR: More Secure Generative Steganography with High-Quality Revealed Secret Images (ICML 2025 · similarity 0.85)
- STLC-KG:A Social Text Steganalysis Method Combining Large-Scale Language Models and Common-Sense Knowledge Graphs (AAAI 2025 · similarity 0.84)
- Edit Distance Robust Watermarks via Indexing Pseudorandom Codes (NeurIPS 2024 · similarity 0.82)
- Adaptive Text Watermark for Large Language Models (ICML 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.