Counting Number of Parameters in Neural Networks

Note: This is an academics based problem.

So in a recent in-class quiz, we were asked that if we have an input layer consisting of 20 nodes along with 2 hidden layers (one of size 10 and the other of size 5), what will the total number of parameters in this network? How can we compute this?

Additionally, how do we know what shapes are they weights of? How can we determine which activation functions are suitable for such a neural network?

My idea was that (20*10) + (10*5) + (biases = 10+5) = 265. So 265 should be the number of parameters. For shapes/activation functions, from what I understand, it just depends on the data, no? Couldn't think of any way to directly predict it from this limited information

Topic neural activation-function weighted-data neural-network social-network-analysis

Category Data Science

1
answered at

Actiavation function isnt a parameter.

But here is general formula for counting weghts:

Suppose for neural network with two hidden layers, inputs dimension is "I", Hidden number of neurons in Layer 1 is "H1", Hidden number of neurons in Layer 2 is "H2" And number of outputs is "O"

weights = (I+1)*H1 +(H1+1)*H2 +(H2+1)*O

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.