How to compare a machine learning model and a rule based model

The difference between machine leaning models and rule based model is that you feed input and output to machine learning models to get rules where you feed input and rules to rule based models to get output.

Say you have a dataset with 10k records. You might use 8k of them to build a machine learning model and the rest to validate your model. On the other hand, you would probably use all the 10k records to create a rule based model. If this was a classification model, how should I compare the 2 models? Accuracy of the machine learning model from the test dataset (2k) and accuracy of the rule based model from all the records?

Topic validation machine-learning

Category Data Science

1
answered at

For the rule base model I would take the exact same approach as for ML model, split the data into train and test set since at the end you want to check whether or not your rule base and ml models generalize well

It is also recommended to used the same sets to train and evaluate both models in such a way that it is a fair comparison.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.