Research map / Large language models / Language, safety and interpretability
OSCS: Online Selection with Provable FAR Control for LLM Safety
Zirui Hu, Zheng Zhang, Yingjie Wang, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SCM-md with OSCS-md for FAR control | SCM-md · Yelp BadNets, BERT-base | FAR: 0.090 -> 0.028 | Improved |
| replace SCM-badacts with OSCS-badacts | SCM-badacts · Yelp BadNets, BERT-base | FAR: 0.116 -> 0.009 | Improved |
| replace CUBE with OSCS-md | CUBE · Yelp BadNets, BERT-base | FAR: 0.322 -> 0.028 | Improved |
| replace SCM-md with OSCS-md on RoBERTa | SCM-md · Yelp BadNets, RoBERTa-base | FAR: 0.209 -> 0.024 | Improved |
| replace SCM-md with OSCS-md in jailbreak detection | SCM-md · AdvBench AutoDAN-ga, Mistral-7B | FAR: 0.142±0.003 -> 0.044±0.006 | Improved |
| replace SCM-ppl with OSCS-ppl in jailbreak detection | SCM-ppl · AdvBench AutoDAN-ga, Llama-3.1-8B | FAR: 0.159±0.003 -> 0.011±0.006 | Improved |
| replace SCM-md with OSCS-md on Qwen2 jailbreak | SCM-md · AdvBench AutoDAN-ga, Qwen2-7B | FAR: 0.000±0.000 vs 0.047±0.001 | No clear effect |
| replace SCM-ppl with OSCS-ppl: detection power cost | SCM-ppl · AdvBench AutoDAN-ga, Mistral-7B | Power: 0.899 -> 0.711 | Hurt |
| replace SCM-badacts with OSCS-badacts: power cost on Yelp BadNets | SCM-badacts · Yelp BadNets, BERT-base | Power: 0.995 -> 0.853 | Hurt |
| lower FAR threshold alpha to 0.01 | OSCS · Backdoor detection, BERT-base | FAR | No clear effect |
| run OSCS over longer stream (more observations) | OSCS · Yelp RoBERTa-base, over time | FAR | Improved |
| swap scoring function from MDS to BadActs in OSCS | OSCS · Backdoor detection | FAR | No clear effect |
Closest papers by meaning
- Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs (NeurIPS 2026 · similarity 0.87)
- On Calibration of LLM-based Guard Models for Reliable Content Moderation (ICLR 2025 · similarity 0.87)
- Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks (ICLR 2025 · similarity 0.87)
- Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models (ICLR 2025 · similarity 0.86)
- MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability (NeurIPS 2024 · similarity 0.86)
- Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
