Research map / Label-efficient and robust learning / Self-supervised and contrastive
PARP: Prune, Adjust and Re-Prune for Self-Supervised Speech Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply 10% PARP pruning to wav2vec2-base | wav2vec2-base · Librispeech 10min, no LM | WER: 49.3/53.2 -> 37.69/43.66 | Helped |
| Apply 10% PARP pruning to wav2vec2-base | wav2vec2-base · Librispeech 10min with 4-gram LM | WER: 27.82/32.02 -> 25.17/32.13 | Mixed |
| Apply 10% PARP pruning to wav2vec2-base | wav2vec2-base · Librispeech 10min with transformer LM | WER: 27.16/32.68 -> 25.45/32.46 | Helped |
| Use PARP rather than regular cross-task transfer | BERT · GLUE CoLA | accuracy: 10.12 -> 28.48 | Helped |
| Use PARP rather than regular cross-task transfer | BERT · GLUE STS-B | accuracy: 80.27 -> 86.25 | Helped |
| Use PARP rather than regular cross-task transfer | XLNet · GLUE CoLA | accuracy: 11.47 -> 30.09 | Helped |
| Start PARP from a random-pruning mask instead of an MPI mask (ablation) | PARP · Low-resource English ASR | WER | Hurt |
| Start PARP from a high-sparsity MPI mask rather than a lower-sparsity mask progressively pruned to target (ablation) | PARP · Librispeech low-resource ASR, high sparsity | WER: up to over 50% absolute WER reduction | Hurt |
| Vary the PARP pruning frequency (ablation) | PARP · Downstream ASR | WER | No clear effect |
| Use different IMP rewinding starting points (ablation) | IMP · Downstream ASR | WER | No clear effect |
| Transfer an OMP mask across languages and adapt it with PARP rather than regular finetuning | wav2vec2 in H2L · Cross-lingual ASR mask transfer | PER | Helped |
| Jointly finetune one shared model for 10 languages rather than finetuning each language separately (ablation) | PARP shared subnetwork · H2L and CSR | PER | Mixed |
Closest papers by meaning
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.88)
- Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models (NeurIPS 2024 · similarity 0.86)
- AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models (NeurIPS 2024 · similarity 0.86)
- CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processing (NeurIPS 2024 · similarity 0.86)
- Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.86)
- Pruning Large Language Models with Semi-Structural Adaptive Sparse Training (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.