Research map / Architectures and efficiency / RNNs and normalization
MLlib: Machine Learning in Apache Spark
Xiangrui Meng, Joseph Bradley, Burak Yavuz, Evan Sparks, Shivaram Venkataraman, Davies Liu, Jeremy Freeman, DB Tsai, Manish Amde, Sean Owen, Doris Xin, Reynold Xin and 4 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| upgrade from MLlib 1.0 to MLlib 1.1 (algorithmic and communication improvements) | MLlib · spark-perf synthetic datasets, 16 workers | speedup: 3x speedup on average | Improved |
Closest papers by meaning
- Apache Mahout: Machine Learning on Distributed Dataflow Systems (JMLR 2020 · similarity 0.85)
- Predicting Machine Learning Pipeline Runtimes in the Context of Automated Machine Learning (TPAMI 2021 · similarity 0.84)
- mlr3pipelines - Flexible Machine Learning Pipelines in R (JMLR 2021 · similarity 0.83)
- Pfeife: Automatic Pipeline Parallelism for PyTorch (ICML 2025 · similarity 0.83)
- Croissant: A Metadata Format for ML-Ready Datasets (NeurIPS 2024 · similarity 0.82)
- ADELA: Accelerating Evolutionary Design of Machine Learning Pipelines with the Accompanying Surrogate Model (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
