Detect change in performance of new rasa model

I have a rasa nlu model running to detect intents and entities. Whenever a new model is loaded by rasa after training, I want to know how good or bad it is from previous model.

What I am planning is , whenever a model is loaded first time, I can test it against some gold standard data and save the output metrics to compare with previous saved metrics.

Questions:

  1. Is there any event which is loaded first and one time only when a model is loaded by rasa nlu? I can write test and save metric logic in that event.
  2. Do you have a better alternate suggestion to achieve this purpose?

Topic rasa-nlu performance

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.