ResNet50 Overfitting even after Dropout

I have a dataset with 60k images in three categories i.e nude, sexy, and safe (each having 30k Images). I am using ResNet50 and observed that the training accuracy and validation accuracy is ok (around 0.82-0.88) although, the validation loss fluctuates a bit.

But, on testing, the precision and recall are too low for each of the classes, around(0.30, 0.25)

I have tried the following things:

  1. Adding DropOut: I have tried adding DropOut with different rates (0.2 - 0.8)
  2. Freezing and unfreezing batchNorm layers.
  3. Using Inception Network
  4. Adding data augmentation (with and without augmentation)

Here is the code

1. Model

model = Sequential()
base_model =ResNet50(include_top = False, pooling = RESNET50_POOLING_AVERAGE)

base_model.trainable = False
# un-freeze the BatchNorm layers
for layer in base_model.layers:
    if BatchNormalization in layer.__class__.__name__:
        layer.trainable = True
        
model.add(base_model)
#model.add(Dropout(0.7))
model.add(Dense(512, activation = 'relu'))
#model.add(Dropout(0.8))

model.add(Dense(3, activation = DENSE_LAYER_ACTIVATION))
sgd = keras.optimizers.SGD(lr = 0.001, decay = 1e-6, momentum = 0.9, nesterov = True)
model.compile(optimizer = sgd, loss = OBJECTIVE_FUNCTION, metrics = LOSS_METRICS)
model.summary()

2. Data Generator

NUM_CLASSES = 3

# Fixed for Cats  Dogs color images
CHANNELS = 3

IMAGE_RESIZE = 224
RESNET50_POOLING_AVERAGE = 'avg'
DENSE_LAYER_ACTIVATION = 'softmax'
OBJECTIVE_FUNCTION = 'categorical_crossentropy'

# Common accuracy metric for all outputs, but can use different metrics for different output
LOSS_METRICS = ['accuracy']

# EARLY_STOP_PATIENCE must be  NUM_EPOCHS
NUM_EPOCHS = 10
EARLY_STOP_PATIENCE = 3

# These steps value should be proper FACTOR of no.-of-images in train  valid folders respectively
# NOTE that these BATCH* are for Keras ImageDataGenerator batching to fill epoch step input
BATCH_SIZE_TRAINING =BATCH_SIZE_VALIDATION= BATCH_SIZE

# Using 1 to easily manage mapping between test_generator  prediction for submission preparation
BATCH_SIZE_TESTING = 1

image_size = IMAGE_RESIZE
data_generator = ImageDataGenerator( rescale=1./255,
                              horizontal_flip=True,
                              vertical_flip=True,
                              validation_split=0.2,
                              preprocessing_function = preprocess_input)
train_generator = data_generator.flow_from_directory(
        'Data/training/',
        target_size=(IMAGE_RESIZE, IMAGE_RESIZE),
        batch_size=BATCH_SIZE,
        shuffle= True,
        class_mode='categorical')

validation_generator = data_generator.flow_from_directory(
        'Data/validation/',
        target_size=(IMAGE_RESIZE, IMAGE_RESIZE),
        batch_size=BATCH_SIZE,
        shuffle= True,
        class_mode='categorical') 

3. Fit

import keras.backend as K
K.set_learning_phase(1)
fit_history = model.fit_generator(
        train_generator,
        steps_per_epoch=num_train_steps,
        epochs = NUM_EPOCHS,
        validation_data=validation_generator,
        validation_steps=num_valid_steps,
        callbacks=[cb_checkpointer,cb_early_stopper]
)

Topic inceptionresnetv2 keras tensorflow

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.