What is the problem that causes overfitting in the code?

**

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from keras import models
from keras.layers import Dense
from keras.regularizers import l1
from keras.layers import Activation
from keras.layers import Dropout
from sklearn.preprocessing import StandardScaler
std=StandardScaler();
x_train, x_test, y_train, y_test=train_test_split(features,target,test_size=0.2,stratify=target,random_state=1)
X_train_std=std.fit_transform(x_train)
X_test_std=std.transform(x_test)
network = models.Sequential()
network.add(Dropout(0.2, input_shape=(55,)))
network.add(Dense(units=16, activation='linear', activity_regularizer=l1(0.0001)))
network.add(Activation('relu'))
network.add(Dropout(0.2))
network.add(Dense(units=32, activation='linear', activity_regularizer=l1(0.0001)))
network.add(Activation('relu'))
network.add(Dropout(0.2))
network.add(Dense(units=1, activation='sigmoid'))
network.compile(loss=binary_crossentropy,optimizer=adam,metrics=[accuracy])
history=network.fit(X_train_std,y_train,epochs=100,batch_size=10,validation_data=(x_test, y_test))

**

Topic overfitting dropout regularization neural-network machine-learning

Category Data Science


Based on your comment, I would like to point out a few things.

1.) First of all Neural Nets (or Deep Learning in general) is used when we have a large amount of data. The data you have is very small and Neural Net needs more data to learn from it. You would be much better off with using traditional machine learning models instead of deep learning.

2.) Since your data is imbalanced, accuracy is the worst choice of metric that your could be using. This is the reason your are getting high accuracy. Using accuracy in imbalanced data will always result in high accuracy because the model will always predict the majority class and hence will give high accuracy. You should use other metrics like f1 score and others. A simple google search would give you the answer.

3.) The reason you are getting constant validation loss might have something to do with the way you modeled your neural net. I once had the same issue but I don't remember how I solved it. Most probably it had something to do with the activation function or the number of neurons but it could be different in your case. Try different activation functions and no of neurons to see if something works. If it works out then do let me know here what the problem was!

Cheers!

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.