Is it correct to train and validate the model on F1-score metrics?

I am trying to do experiments on multiple data sets. Some are more imbalanced than others. Now, in order to assure fair reporting, we compute F1-Score on test data. In most machine learning models, we train and validate the model via accuracy measure metric. However, this time, I decided to train and validate the model on an F1-score metric measure. Technically, there should be no problems, in my opinion. However, I am wondering if this is the correct approach to go.

Second, when I use this method (training, validation on F1-score), I receive a higher loss error and a lower F1-score on training data than on validation data. I’m not sure why.

Topic f1score training deep-learning classification time-series

Category Data Science

1
answered at

In case of imbalanced class, Accuracy should never be used. As a model can just say every observation belongs to majority class and still gets very high accuracy.

For evaluating imbalanced class we use Precision, Recall or F1-score based on whatever metrics align best with out business objective. If you are getting low F1-Score it means that model is not able to perform well. To tackle the low F1-score please try following :

Sampling : Try to balance the data by using UpSampling or downsampling techniques
Class Weights : Use class weights parameter in ML algorithm to assign weights to classes
Data Augmentation : Use data augmentation techniques like SMOTE to generate more examples

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.