Relationship between Sigmoid and Gaussing Distribution

I was reading this article where I came across the following statement in the context of "Why do we use sigmoid activation function in Neural Nets?":

The assumption of a dependent variable to follow a sigmoid function inherently assumes a Gaussian distribution for the independent variable which is a general distribution we see for a lot of randomly occurring events and this is a good generic distribution to start with.

Could someone elaborate on this relationship between the two?

Topic neural activation-function gaussian

Category Data Science

1
answered at

Yes sure, the core idea is that the sigmoid function approximate very nicely the cumulative density function of a Gaussian distribution. Therefore, if the input of a neuron is a continuous Gaussian distribution, the probability of the output being inferior to a certain value will follow (almost) the shape of the sigmoid function.

A detailed explanation can be found p16 of the following article:

https://www.researchgate.net/publication/220233479_Catastrophic_Forgetting_and_the_Pseudorehearsal_Solution_in_Hopfield-type_Networks

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.