Keywords: machine, learning, mlflow, mleap, spark, model, serving, deployment, production, webapp
Successfully productionizing machine learning models is difficult. In addition to providing sufficient accuracy, deployed models often need to meet strict latency requirements. This requires infrastructure for monitoring the statistical and system performance of deployed models, as well as high-performance frameworks for model inference.
MLflow is an open source machine learning platform that aims to optimize the machine learning lifecycle, providing useful services such as model tracking and reproducible training sessions. In particular, MLflow addresses the challenges of managing and monitoring production model deployments. We will demonstrate how MLflow can be used in conjunction with the low-latency MLeap inference framework to efficiently deploy Spark models while meeting near-real-time latency SLAs.