Keras: Very high loss for Autoencoder

I am trying to implement an autoencoder for prediction of multiple labels using Keras. This is a snippet:

input = Input(shape=(768,))
hidden1 = Dense(512, activation='relu')(input)
compressed = Dense(256, activation='relu', activity_regularizer=l1(10e-6))(hidden1) 
hidden2 = Dense(512, activation='relu')(compressed)
output = Dense(768, activation='sigmoid')(hidden2) # sigmoid is used because output of autoencoder is a set of probabilities

model = Model(input, output)
model.compile(optimizer='adam', loss='categorical_crossentropy') # categorical_crossentropy is used because it's prediction of multiple labels
history = model.fit(x_train, x_train, epochs=100, batch_size=50, validation_split=0.2)

I ran this in Jupyter Notebook (CPU) and I am getting loss and validation loss as: loss: 193.8085 - val_loss: 439.7132

but when I ran it in Google Colab (GPU), I am getting very high loss and validation loss: loss: 28383285849773932.0000 - val_loss: 26927464965996544.0000.

What could be the reason for this behavior?

Topic colab validation keras autoencoder loss-function

Category Data Science


You should not use the categorical cross-entropy loss, but the binary cross-entropy.

The categorical cross-entropy is meant for categorical probability distribution. In your scenario, this means that only one label would be active at the same time. If multiple labels can be active at the same time, then they are independent, and you need to compute the binary cross-entropy.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.