How are the confidence intervals of a model interpreted?

I am doing some work with R and after obtaining the confusion matrix I have obtained the following metrics corresponding to a logistic regression:

Accuracy : 0.7763         
                 95% CI : (0.6662, 0.864)
    No Information Rate : 0.5395         
    P-Value [Acc  NIR] : 1.629e-05

And it is not clear to me how CI would be interpreted.

Maybe it would be that the Accuracy can take values between 0.666 and 0.864? What does it mean that the CI are so large?

If someone could clarify it to me I would appreciate it. Best regards.

Topic confidence machine-learning-model rstudio evaluation machine-learning

Category Data Science

1
answered at

The other answer is correct and summed it up nicely: we are 95% confident the accuracy value will fall somewhere between .666 and .864. It is a probability claim for how representative your number is.

To your other question, it could mean a couple different things based on your data what the CI means. In general, the larger the CI, the bigger the range for your numbers will be. For example, we can be 95% certain the accuracy falls between .666 and .864, but you may change the CI to 99% and it might give you a value like .333 and 1.264 or something.

When you have a large range in your CI, it usually means you have high variability in your data (some of your datapoints are around .666 while others are around .864). The more data you have and the more related the data is, the lower the range will become.

It depends for what purpose you are crunching these numbers for, but generally speaking, Higher confidence intervals like 95% give you more certainty in the data while a smaller confidence interval like 75% can compile more neatly digestible graphs while sacrificing some accuracy.

Hope this helps a bit! :)

1
answered at

It tells you that with 95% probability the accuracy is estimated to lie between those bounds. Thus, when repeating the experiment very often, computing the accuracies of each individual run, you would obtain values of the accuracies outside these bounds in only 5% of the cases.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.