Confusion matrix, when mistakes below diagonal are better than above the diagonal

I have a classification problem and I am producing a confusion matrix. Ideally one wants to get all results in the diagonal. I get quite many points around diagonal for different algorithms. Still for my use-case I want to favor algorithms that underpredict the class (I have ordinal data) and not overpredict.

Is there a metric that can measure under and overprediction and rate those errors with a different weight? The typical accuracy, precision terms assume that all mistakes are the same.

Of course I can try to implement my own metric but I am quite sure that I am not the first that is having this issue.

Any metric available that you know already? Thanks Alex

Topic confusion-matrix classification

Category Data Science

1
answered at

You are incorrect when you say accuracy and precision assume the same mistakes. They are quite different in nature as to when they are applied for separate use-cases.

For more details on how they are different - https://towardsdatascience.com/accuracy-precision-recall-or-f1-331fb37c5cb9

As per your situation, you could opt for precision as you are looking only for the right answers, in case of a spam email detection.

There is no metric as such which calculates how much your model underfits or overfits but a genera idea can be made when you compare the validation curve and the training curve.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.