MLflow real world experience

Can someone provide a summary of the real world deployment experience of MLflow? We have a few ML models (e.g., LightGBM, tensorflow v2, etc.) and want to avoid framework like SageMaker (due to customer requirement). So we are looking into various ways of hosting ML models for inferencing. Latency is one key performance metric that is very important to us. MLflow looks like a good choice for us.

It would be greatly appreciated if the users of MLflow can share their experience.

Topic mlops mlflow

Category Data Science


I am currently using MLflow together with Databricks at my company, and up to now, I would say it is quite ok for managing models versions, tracking models metrics and building the models workflow from a dev/experimenting/training status to production. I am not sure if it can be considered a standalone tool for that, meaning that you should combine it with something like Azure Databricks (what I currently use). It's nice the way you can customize the tags, metrics or artifacts that you might want to keep logged after each run or model training, as follows: enter image description here

where you can also visualize several run experiments as:

enter image description here (these models are for the shake of the example, so metric values are not optimized) and storing the generated models files (e.g. a pickle file) in the corresponding location: enter image description here

Once it is generated, you can register your best model via the databricks UI (in case you finally use it) or via mlflow code as follows (here I register the last one, you should take the one with the best prefered metric which mlflow also gives you directly as an option): enter image description here

so as to be loaded later on and make predictions with your model. Once you your are happy with some model, you can transition it: enter image description here

Another nice feature I like is that it offers you custom modules for the most used frameworks like scikitlearn, xgboost, keras, etc

and build a container image, deploy it... but this is another longer topic.

Of course, you can also use MLflow with another third party or even locally.

About the performance point, do you mean the inference itself or some other source of latency? You can make inferences with your model via SPARK UDFs (PySPARK in my case) info, in case you need several predictions as fast as possible or simply for batch inference.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.