Learning rate Scheduler
A very important aspect in deep learning
is the learning rate
. Can someone tell me, how to initialize the lr and how to choose the decaying rate. I'm sure there are valuable pointers that some experienced people in the community can share with others. I've noticed that many choose to do a custom scheduler rather than use available ones.
Can someone tell me why and what influences the change in the lr? And when to describe a lr as being small, medium or large? I want to understand it enough to actually make sound choices. Thank you kind souls. I appreciate this community very much.
Topic learning-rate deep-learning machine-learning
Category Data Science