Steps taking too long to complete

I'm trying to train a model which in my opinion is taking too long compared to other datasets given that it's taking about 9s to complete a step. I think that the problem is because the dataset is not being stored on ram, but I'm not sure of this.

The code is the following:

def load_data():

    train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True)
    train_generator = train_datagen.flow_from_directory(path1, target_size=(200, 200), batch_size=32, class_mode="binary")

    test_datagen = ImageDataGenerator(rescale=1./255)
    test_generator = test_datagen.flow_from_directory(path2, target_size=(200, 200),batch_size=32, class_mode="binary")

    return train_generator, test_generator

Model:

  • Sequential model
  • 2 Convolutional layers with 32 neurons, activation = relu.
  • 1 Convolutional layer with 64 neurons, activation = relu.
  • Flattening and Dense layer, activation = relu.
  • Dropout of 0.5
  • Output layer (Dense) with sigmoid activation.
  • Adam optimizer.
  • Loss: binary cross entropy.

Fit:

model.fit_generator(x, steps_per_epoch=37, epochs=50, validation_data=y, validation_steps=3, callbacks=[tensorboard])
  • My dataset has 1201 images and 2 classes.
  • I built the model following this tutorial.
  • My GPU is a GTX 1060 3gb.
  • 8gb of ram.
  • The images are being reshaped to 200x200.

If you could help me I'd appreciate it. Thank you very much!

Topic keras tensorflow neural-network python

Category Data Science


Your images are probably too large for using just three convolutions. So the number of parameters is huge during flattening (and the net cannot learn global features of the images, but this is a different problem).

Try to

  • use more convolution and pooling operations

  • see if strided convolutions help

  • check the number of parameters using the summary method of Keras

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.