Research map / Label-efficient and robust learning / Knowledge distillation
Measuring and Reducing Model Update Regression in Structured Prediction for NLP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use backward-congruent re-ranking instead of untreated updating | Dependency parsers · EWT, three update settings | NFI: an average of 58% relative reduction in NFI | Helped |
| Use backward-congruent re-ranking instead of untreated updating | TOP seq2seq parsers · TOP, three update settings | NFI: 73% relative NFI reduction on average | Helped |
| Use prediction ensembling rather than untreated updating | Dependency parsers · EWT, three update settings | NFI: 28% relative reduction in NFI | Helped |
| Use prediction ensembling rather than untreated updating | TOP seq2seq parsers · TOP, three update settings | NFI: 28% relative NFI reduction on average | Helped |
| Distill the old model's predictions into the new model | Dependency parsers · EWT, three update settings | NFI | No clear effect |
| Distill the old model's predictions into the new model | TOP seq2seq parsers · TOP, homogeneous update settings | NFI | Helped |
| Distill the old model's predictions into the new model | TOP seq2seq parsers · TOP, s2s-base→s2s-large | NFI: 24.40 to 15.97 | Helped |
| Generate candidates with dropout-p sampling rather than beam search or truncated sampling (ablation) | BCR · TOP, s2s-base→s2s-large | NFI: the gaps in NFI are larger than 10 points | Helped |
| Use top-p or top-k sampling for candidate generation (ablation) | BCR · Structured prediction tasks | NFI | Hurt |
| Increase dropout-p from 0.1 to 0.3 (ablation) | BCR · Structured prediction tasks | NFI: 0.1 to 0.3 | Helped |
| Increase dropout-p from 0.1 to 0.3 (ablation) | BCR · Structured prediction tasks | accuracy: 0.1 to 0.3 | Hurt |
| Increase the number of BCR candidates (ablation) | BCR · TOP, s2s-base→s2s-large | NFI | Helped |
Closest papers by meaning
- Aligning Language Model Benchmarks with Pairwise Preferences (NeurIPS 2026 · similarity 0.84)
- Automating Dataset Updates Towards Reliable and Timely Evaluation of Large Language Models (NeurIPS 2024 · similarity 0.84)
- Wasserstein Distance Constraint and Parameter Sparsification for Batched and Iterative Knowledge Editing (AAAI 2025 · similarity 0.83)
- Language Model Cascades: Token-Level Uncertainty And Beyond (ICLR 2024 · similarity 0.82)
- Data Efficient Adaptation in Large Language Models via Continuous Low-Rank Fine-Tuning (NeurIPS 2025 · similarity 0.82)
- Sample then Identify: A General Framework for Risk Control and Assessment in Multimodal Large Language Models (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.