Research map / Adversarial robustness and security / Attacks on text and graphs
RETVec: Resilient and Efficient Text Vectorizer
Elie Bursztein, Marina Zhang, Owen Skipper Vallis, Xinyu Jia, Alexey Kurakin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RETVec instead of fastText | Classification models on MASSIVE · MASSIVE | accuracy: 16.7% -> 73.0% | Improved |
| Use RETVec instead of SentencePiece | BERT-Base · GLUE dev sets | Avg: 79.6 -> 79.1 | Hurt |
| Use RETVec instead of fastText | BERT-Mini · AG News character-level attacks | attack success rate: 56.3% -> 44.8% | Improved |
| Add the pre-trained embedding model to RETVec-raw | RETVec classification models · Classification benchmarks | accuracy | Improved |
| Add the pre-trained embedding model to RETVec-raw | RETVec classification models · Typo-augmented test sets | accuracy | Improved |
| Use a 256-dimensional embedding rather than 512 dimensions (ablation) | RETVec · Multilingual Amazon Reviews (Polarity) | accuracy: 92.9% -> 92.9% | No clear effect |
| Replace the simple MLP with MLP + BERT (ablation) | RETVec · Multilingual Amazon Reviews (Polarity) | accuracy: 92.9% -> 92.5% | Hurt |
| Increase maximum word length from 16 to 24 characters (ablation) | RETVec · Multilingual Amazon Reviews (Polarity) | accuracy: 92.9% -> 92.9% | No clear effect |
| Remove spatial dropout (ablation) | RETVec · Multilingual Amazon Reviews (Polarity) | accuracy: 92.9% -> 92.4% | Hurt |
| Add augmentation-position prediction to similarity learning (ablation) | RETVec pre-training · Multilingual Amazon Reviews (Polarity) | accuracy: 92.9% -> 92.7% | Hurt |
| Use a more complex architecture than the simple MLP (ablation) | RETVec pre-training · Multilingual Amazon Reviews (Polarity) | accuracy: 92.9% -> 92.5% | Hurt |
Closest papers by meaning
- RETSim: Resilient and Efficient Text Similarity (ICLR 2024 · similarity 0.84)
- Repurposing Language Models into Embedding Models: Finding the Compute-Optimal Recipe (NeurIPS 2024 · similarity 0.81)
- Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models (ICLR 2025 · similarity 0.81)
- MMTEB: Massive Multilingual Text Embedding Benchmark (ICLR 2025 · similarity 0.80)
- The Scandinavian Embedding Benchmarks: Comprehensive Assessment of Multilingual and Monolingual Text Embedding (NeurIPS 2024 · similarity 0.80)
- GuardT2I: Defending Text-to-Image Models from Adversarial Prompts (NeurIPS 2024 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
