What does it mean if performance of two different iterations of the same network (CNN model) varies a lot?

Question

What does it mean if performance of two different iterations of the same network (CNN model) varies a lot?

Jumabek Alikhanov

2021年8月4日 11:08

So I trained CNN model for people detection on caltech-pedestrian dataset: Then I was curious and evaluated the model in every 1000th iteration on Evaluation toolbox(I guarantee, there is no bug in evaluation).

However, plot of the performance does not look so good. The miss rate spikes between 20K(20,000) and 30K iterations.

I am confused what does that mean. I mean usually we would expect the miss rate to decrease as we train the model more.

I am using yolo object detection network https://github.com/AlexeyAB/darknet/blob/master/cfg/tiny-yolo-voc.cfg

So learning options are the same as https://github.com/AlexeyAB/darknet/blob/master/src

I also tried 10x smaller learning rate. However, no luck:

Topic convolutional-neural-network object-recognition machine-learning

Category Data Science

Brian Spiering · Accepted Answer · 2021年7月3日 20:22

1

Brian Spiering answered at 2021年7月3日 20:22

This is a relatively common phenomena called double descent.

What does it mean if performance of two different iterations of the same network (CNN model) varies a lot?

About