Research map / Adversarial robustness and security / Attacks on text and graphs
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use adversarial instead of benign system prompt | GPT-4 · RealToxicityPrompts nontoxic | toxicity probability: 0.3% -> 100% | Hurt |
| use adversarial instead of benign system prompt | GPT-3.5 · RealToxicityPrompts nontoxic | expected maximum toxicity: 0.13 -> 0.76 | Hurt |
| instruction tuning and RLHF (GPT-3.5/4 vs GPT-3 Davinci) | GPT models · RealToxicityPrompts nontoxic | toxicity probability: 31% -> 0.3% | Helped |
| GPT-4 instead of GPT-3.5 under adversarial system prompt | GPT models · RealToxicityPrompts nontoxic | expected maximum toxicity: 0.76 -> 0.91 | Hurt |
| use LLM-generated (GPT-4) toxic task prompts instead of RealToxicityPrompts toxic | GPT-4 adversarial · toxic prompts | expected maximum toxicity: 0.94 -> 0.95 | Hurt |
| use LLM-generated toxic prompts under benign system prompt | GPT-4 · toxic prompts | expected maximum toxicity: 0.38 -> 0.24 | Helped |
| more instructive task description (Template 2) instead of Template 1 (ablation) | GPT-4 · AdvGLUE | average robust accuracy: 78.18 -> 74.75 | No clear effect |
| add system prompt noting adversarial attacks exist (Template 3) (ablation) | GPT-4 · AdvGLUE | average robust accuracy: 78.18 -> 78.41 | No clear effect |
| GPT-4 instead of GPT-3.5 | AdvGLUE · AdvGLUE | average robust accuracy: 67.37 -> 78.41 | Helped |
| GPT-4 instead of GPT-3.5 | BBNLI · BBNLI | mean bias score (NLI): 0.048 -> 0.071 | Hurt |
Closest papers by meaning
- VMDT: Decoding the Trustworthiness of Video Foundation Models (NeurIPS 2025 · similarity 0.87)
- Position: TrustLLM: Trustworthiness in Large Language Models (ICML 2024 · similarity 0.86)
- MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models (NeurIPS 2024 · similarity 0.86)
- Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compression (ICML 2024 · similarity 0.86)
- More RLHF, More Trust? On The Impact of Preference Alignment On Trustworthiness (ICLR 2025 · similarity 0.85)
- U-trustworthy Models. Reliability, Competence, and Confidence in Decision-Making (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.