What causes explosion in MSE when training?

I (probably) well overfitted/overtrained a model. But I was just curious as to what might cause this type of behaviour. I carried on training (Epoch 1/50 is not the first epoch of training this model). You can see the mse (loss) is v low. It slowly decreases over epochs 1-40. Then soon it explodes. What causes this type of behaviour when training models?

55706/55706 [=======] - 109s 2ms/step - loss: 0.0059 - coeff_determination: 0.9688
…
Epoch 5/50
55706/55706 [=======] - 105s 2ms/step - loss: 0.0033 - coeff_determination: 0.9828
...
Epoch 40/50
55706/55706 [=======] - 89s 2ms/step - loss: 0.0018 - coeff_determination: 0.9906
Epoch 41/50
55706/55706 [=======] - 110s 2ms/step - loss: 0.1853 - coeff_determination: 0.0299
Epoch 42/50
55706/55706 [=======] - 97s 2ms/step - loss: 0.3120 - coeff_determination: -0.6346
Epoch 43/50
55706/55706 [=======] - 99s 2ms/step - loss: 0.3120 - coeff_determination: -0.6345```

Topic mse machine-learning-model overfitting training machine-learning

Category Data Science

1
answered at

This behaviour might be related to the learning rate, first getting you closer to the loss' minimum, then overshoots it. Consider different learning rates, including the adaptive types.

Also, try different regularization methods.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.