In neural networks model, which number of hidden units to select?

Question

In neural networks model, which number of hidden units to select?

coding_ninza

2020年5月4日 08:05

In the neural networks model, how many numbers of hidden units need to keep to get an optimal result, as per Cybenko theorem which demonstrates that only one hidden layer is sufficient to solve any regression/classification problem but the selection of the number of units in a hidden layer is very important because it impacts the model performance. Is there a theory to tell us how to choose the optimal number of units for a hidden layer?

Topic ann deep-learning neural-network machine-learning

Category Data Science

Leevo · Accepted Answer · 2020年5月4日 08:05

Unfrtunately not, there is not theory to tell us what is the right number of units to choose. As there is not theory for the number of hidden layers to choose. On this respect, Deep Learning is still more an art than a science.

It's true that with one hidden layer we could theoretically solve any problem, but most of the problems are complicated enough to require unimaginable amounts computation.

I think in the end it all boils down to two main issues:

Your specific task, i.e. how large and deep a Network must be so that your model works as you need.
The compute power at your disposal.

When it comes to this, I strongly suggest to prioritize depth over width. Deeper Networks (the ones with many hidden layers) are much more efficient and powerful than large Networks (the ones with fewer, larger layers). It seems they are better at producing abstractions on the input data, to transform and process the signal in more sophisticated ways.

In neural networks model, which number of hidden units to select?

About