Store preprocessing function along with model in mlflow.keras
The following is a simplified code snipet that is relevant to storing keras LSTM models in MLFlow.
with mlflow.start_run() as run:
mlflow.keras.log_model(model,lstm)
mlflow.log_params(model_parameters)
mlflow.log_metrics(model_metrics)
However, suppose that for each model there is a corresponding data preprocessing function that need be applied to new data before prediction.
processed_data = custom_processing_function(new_data)
predictions = model.predict(processed_data)
Because each model may have a different preprocessing function, I want to keep track of each pair of the form (preprocessing function, model). Ideally, I am looking for something like that:
with mlflow.start_run() as run:
mlflow.keras.log_model(model,lstm)
mlflow.log_params(model_parameters)
mlflow.log_metrics(model_metrics)
mlflow.log_function(custom_preprocessing) #---------
Is it possible to store preprocessing function in mlflow, and what is a standard or appropriate way to do it?
During the prediction step, how can I call the stored preprocessed function on new data?
Topic mlflow experiments
Category Data Science