Matrix factorization how to initialize weights and biases?

Question

Matrix factorization how to initialize weights and biases?

Karol

2022年2月21日 17:04

I have a matrix factorization and I'm wondering how I should initialize its weights and biases.

When getting prediction (recommendation), after computing a dot product and adding bias I want to use sigmoid function on that to get value from 0 to 1.

But when introducing a sigmoid here I also introduce a possibile vanishing/exploding gradient problem. For that I think that weights can be initialized with xavier function. But what aboud biases? Should I just use uniform distribution from (-0.01, 0.01) for example?

Topic weight-initialization matrix-factorisation deep-learning neural-network machine-learning

Category Data Science

xboard · Accepted Answer · 2021年1月10日 18:34

1

xboard answered at 2021年1月10日 18:34

For matrix factorization I usually see it being initialized by a uniform distribution from [0, 1) like in this (pytorch) or a truncated normal with mean=0.0 and std=1.0 as in this (tensorflow).

Matrix factorization how to initialize weights and biases?

About