Error: An operation has `None` for gradient with categorical_crossentropy
I am trying to train my discriminator network using Keras with the TensorFlow backend. The network is meant to classify the input into one of the 9 output labels. I am passing a 2D input (height, width, no channels) and a one-hot vector for the output. I was able to train the network independently using fit(). However, now that I have switched to train_on_batch, it is giving me the error mentioned.
This is my discriminator code:
def build_discriminator(time_steps, feature_size, input_spectrogram=None):
spectrogram = Input(shape=(time_steps, feature_size))
# spectrogram = tf.placeholder(tf.float32, shape=(None, time_steps, feature_size))
layer0 = Reshape((time_steps, feature_size, 1))(spectrogram)
layer1 = Conv2D(32, kernel_size=(3,3), padding='same')(layer0)
#model.add(LeakyReLU(alpha=0.01))
layer2 = MaxPooling2D(pool_size=(4,4))(layer1)
layer3 = Conv2D(16, kernel_size=(3,3), padding='same')(layer2)
#model.add(LeakyReLU(alpha=0.05))
layer4 = MaxPooling2D(pool_size=(4,4))(layer3)
layer5 = Conv2D(16, kernel_size=(3,3), padding='same')(layer4)
#model.add(LeakyReLU(alpha=0.05))
layer6 = MaxPooling2D(pool_size=(4,4))(layer5)
layer7 = Flatten()(layer6)
layer8 = Dense(16)(layer7)
prediction = Dense(9, activation = 'softmax')(layer8)
# prediction = Dropout(0.1)(layer9)
model = Model(spectrogram, prediction)
opt = optimizers.Adam(lr=0.002, beta_1=0.5)
model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])
return model
This is the code that trains the discriminator:
x_real = batch_x[:half_batch, :, :]
labels_real = batch_labels[:half_batch]
d_loss1, _ = discriminator.train_on_batch(x_real, to_categorical(labels_real, num_classes=9))
# generate 'fake' examples
x_fake, labels_fake = generate_fake_samples(batch_x[half_batch:, :, :], batch_labels[half_batch:], generator)
# update discriminator model weights
d_loss2, _ = discriminator.train_on_batch(x_fake, to_categorical(labels_fake, num_classes=9))
# update the generator via the discriminator's error
g_loss, acc = gan.train_on_batch([batch_x, batch_targets], to_categorical(batch_targets, num_classes=9))
It is throwing the error on this line:
d_loss1, _ = discriminator.train_on_batch(x_real, to_categorical(labels_real, num_classes=9))
The error traceback:
File gan.py, line 126, in train
d_loss1, _ = discriminator.train_on_batch(x_real, to_categorical(labels_real, num_classes=9))
File /home/pallavi/anaconda3/lib/python3.7/site-packages/keras/engine/training.py, line 1513, in train_on_batch
self._make_train_function()
File /home/pallavi/anaconda3/lib/python3.7/site-packages/keras/engine/training.py, line 316, in _make_train_function
loss=self.total_loss)
File /home/pallavi/anaconda3/lib/python3.7/site-packages/keras/legacy/interfaces.py, line 91, in wrapper
return func(*args, **kwargs)
File /home/pallavi/anaconda3/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py, line 75, in symbolic_fn_wrapper
return func(*args, **kwargs)
File /home/pallavi/anaconda3/lib/python3.7/site-packages/keras/optimizers.py, line 504, in get_updates
grads = self.get_gradients(loss, params)
File /home/pallavi/anaconda3/lib/python3.7/site-packages/keras/optimizers.py, line 93, in get_gradients
raise ValueError('An operation has `None` for gradient. '
ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.
I am using TensorFlow-GPU 2.0.0 with Keras 2.3.1. Could someone please help me understand where I am going wrong?
Topic gan training keras tensorflow classification
Category Data Science