Convolutional Neural Network overfitting

Question

Convolutional Neural Network overfitting

Simon

2022年1月24日 14:58

I built a CNN to learn to classify EEG data (only about 4000 training examples, 2 classes, 50-50 class balance). Each training example is 64x512, with 5 channels each

Ive tried to keep the network as simple/small as possible for testing:

ConvLayer (4 filters)
MaxPool
Dropout 50%
Fully connected (50 neurons)
Dropout 50%
Softmax

Im also using weight decay (L2 reg, lambda = 0.001)

The problem is no matter how I play with the filter parameters (size, stride, number) my network keeps overfitting. It fits the training data 100%, but no matter what I do I can't get the test accuracy over 65%.

Why is such a small network overfitting? I thought it was a sample size issue, but I've read a number of research papers on EEG and BCI and they occasionally have even smaller sample sizes than I do

What else can be done to regularize a CNN?

Topic convolutional-neural-network regularization neural-network classification machine-learning

Category Data Science

nimar · Accepted Answer · 2020年5月27日 00:22

One possible solution when you do not have enough data is to use Transfer learning.

This helps you to improve the performance of your model on the test data set. So, you can easily use one of the available pre-trained models in technical literature and update its weights based on your data. Take a look at this video. It is very helpful and you get a lot of ideas how to deal with your problem.

J. O'Brien Antognini · Accepted Answer · 2017年1月5日 18:54

1

J. O'Brien Antognini answered at 2017年1月5日 18:54

Have you tried early stopping? Hold out 20% of the data and stop training when the error on the hold-out set starts to increase. If you can't get high enough accuracy, increase the size of your model until you do.

enterML · Accepted Answer · 2016年12月6日 09:44

This is because you have very little amount of data. If not enough data is provided to CNNs, they are very likely to overfit. You can do the following things in order to overcome this problem:

Data Augmentation : It is a technique to create new examples from the training examples by doing some preprocessing on them e.g. Rotation, Scaling, etc.

Rigorous Dropout : Dropout is a very powerful technique to control overfittting. You can try using more rigorous dropout in your architecture

Convolutional Neural Network overfitting

About