Do non-parametric models always overfit without regularization?

Question

Do non-parametric models always overfit without regularization?

kennysong

2021年3月18日 14:40

Let's scope this to just classification.

It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy down to Bayes error*.

Is this universally true for all non-parametric methods?

*Assuming the model has access to the right features.

Topic non-parametric bias overfitting regularization machine-learning

Category Data Science

Brian Spiering · Accepted Answer · 2021年3月18日 14:40

No - non-parametric methods only means that the method does not assume a function form of the data. There are non-parametric methods such as Random Forest that do not always overfit. In fact nonparametric methods could underfit, it could lack the ability to fit the training data. An example of this would be a decision stump.

Do non-parametric models always overfit without regularization?

About