Research map / Probabilistic and scientific modeling / Molecules and proteins
An Invariant Latent Space Perspective on Language Model Inversion
Wentao Ye, Jiaqi Hu, Haobo Wang, Xinpeng Ti, Zhiqing Xiao, Hao Chen, Liyao Li, Lei Feng, Sai Wu, Junbo Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inverse encoder in front of the frozen LLM decoder (ablation) | Inv2A · User prompt inversion | BLEU: 1.31 -> 35.20 | Improved |
| add inverse encoder instead of LoRA-tuned decoder alone (ablation) | Inv2A · User prompt inversion | BLEU: 26.47 -> 35.20 | Improved |
| use the raw forward LLM as decoder instead of Qwen2 (ablation) | Inv2A · User prompt inversion | BLEU: 33.38 -> 35.20 | Improved |
| add source-aware contrastive learning in alignment phase (ablation) | Inv2A · User prompt inversion | BLEU: 33.91 -> 35.20 | Improved |
| add optional dynamic filter post-refinement (1 round) (ablation) | Inv2A · User prompt inversion | BLEU: 35.20 -> 35.97 | Improved |
| add a second refinement round (ablation) | Inv2A with refine · User prompt inversion | BLEU: 35.97 -> 36.06 | No clear effect |
| naive round-trip decoding of outputs through the LLM | raw LLM as inverse · Alpaca (GPT-2 analysis) | BLEU: BLEU 4.75 | Hurt |
| raise sampling temperature as a defense against inversion | Inv2A vs Output2prompt · User prompt inversion, temperature 0.5 to 2.0 | BLEU: 35.42 -> 30.10 | Mixed |
| inject Gaussian noise into MLP layer of first hidden layer as defense | LLM defense · User prompt inversion | BLEU: ~8% BLEU drop at 2.5e-2 | Mixed |
| reuse LLM latent space with Inv2A instead of training external T5 inverse model | inversion attack · training data scale | BLEU: comparable performance with 20 to 30% of the data | Improved |
| swap Inv2A for Output2prompt on out-of-domain Anthropic HH with LLaMA2-13B | Inv2A · Anthropic HH (LLaMA2-13B) | BLEU: 29.61 -> 30.41 | No clear effect |
Closest papers by meaning
- Better Language Model Inversion by Compactly Representing Next-Token Distributions (NeurIPS 2025 · similarity 0.88)
- Language Model Inversion (ICLR 2024 · similarity 0.87)
- Soft Prompt Recovers Compressed LLMs, Transferably (ICML 2024 · similarity 0.85)
- On the Worst Prompt Performance of Large Language Models (NeurIPS 2024 · similarity 0.85)
- Invariant Features in Language Models: Geometric Characterization and Model Attribution (NeurIPS 2026 · similarity 0.85)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
