Research map / Large language models / AI and society
RedacBench: Can AI Erase Your Secrets?
Hyunjun Jeon, Kyuyoung Kim, Jinwoo Shin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a second adversarial redaction iteration | AR with gpt-5-mini · RedacBench | security score: 63.4 -> 80.9 | Improved |
| add a second adversarial redaction iteration | AR with gpt-4.1-nano · RedacBench | security score: 64.1 -> 61.7 | No clear effect |
| add a second adversarial redaction iteration | AR with gpt-5-mini · RedacBench | utility score: 57.2 -> 37.6 | Hurt |
| use adversarial redaction instead of masking | GPT-4.1-mini · RedacBench | security score: 37.2 -> 53.7 | Improved |
| use adversarial redaction instead of masking | GPT-4.1-mini · RedacBench | utility score: 80.8 -> 68.3 | Hurt |
| process text in sequential chunks (no forward context) (ablation) | AR with gpt-4.1-nano · RedacBench (interactive) | security score: 64.1 -> 55.2 | Hurt |
| use GPT-5 as evaluator instead of GPT-4.1-mini (ablation) | AR (iter 1) outputs of GPT-4.1 · RedacBench | security score: 68.2 -> 92.6 | Mixed |
| apply severity weights to policies (ablation) | Masking with gpt-5 · RedacBench | security score: 38.9 -> 38.3 | Hurt |
| seven AR iterations versus two iterations of larger model | GPT-4.1-mini vs GPT-5 · RedacBench | security-utility trade-off | Improved |
Closest papers by meaning
- Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage (AAAI 2025 · similarity 0.85)
- Unlearned but Not Forgotten: Data Extraction after Exact Unlearning in LLM (NeurIPS 2025 · similarity 0.85)
- Text-Diffusion Red-Teaming of Large Language Models: Unveiling Harmful Behaviors with Proximity Constraints (AAAI 2025 · similarity 0.83)
- Textual Unlearning Gives a False Sense of Unlearning (ICML 2025 · similarity 0.83)
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (ICML 2024 · similarity 0.83)
- Can Sensitive Information Be Deleted From LLMs? Objectives for Defending Against Extraction Attacks (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
