Store preprocessing function along with model in mlflow.keras

The following is a simplified code snipet that is relevant to storing keras LSTM models in MLFlow.

with mlflow.start_run() as run:
    mlflow.keras.log_model(model,lstm)
    mlflow.log_params(model_parameters)
    mlflow.log_metrics(model_metrics)

However, suppose that for each model there is a corresponding data preprocessing function that need be applied to new data before prediction.

processed_data = custom_processing_function(new_data)
predictions = model.predict(processed_data)

Because each model may have a different preprocessing function, I want to keep track of each pair of the form (preprocessing function, model). Ideally, I am looking for something like that:

with mlflow.start_run() as run:
    mlflow.keras.log_model(model,lstm)
    mlflow.log_params(model_parameters)
    mlflow.log_metrics(model_metrics)
    mlflow.log_function(custom_preprocessing) #---------
  1. Is it possible to store preprocessing function in mlflow, and what is a standard or appropriate way to do it?

  2. During the prediction step, how can I call the stored preprocessed function on new data?

Topic mlflow experiments

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.