My goal is to calculate backpropagation(Especially the backpropagation of the bias).
For example, X, W and B are python numpy array, such as [[0,0],[0,1]] , [[5,5,5],[10,10,10]] and [1,2,3] for each. And, suppose dL/dY is [[1,2,3],[4,5,6]].
How to calculate dL/dB? Answer should be [5, 7, 9]. why is it calculated that way?
Topic bias backpropagation
Category Data Science
The image you provided already shows how to calculate the derivative of the loss with regards to the biases, it is equal to the derivative of the loss with regards to the y values. The only thing you have to do is take the sum of $\frac{\partial L}{\partial Y}$ over the number of samples, which gives you $\frac{\partial L}{\partial B} = [1 + 4, 2 + 5, 3 + 6] = [5, 7, 9]$.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.