Seems like the Keras .fit and .evaluate methods give different training accuracy (yet the same validation accuracy). Same thing on loss.?

The code below is a translation of Nielsen's first mnist code to Keras. Surprisingly, the last accuracy value of the .fit method and the accuracy value for the .evaluate method are different for the training data. As I expected, for the validation data the accuracies are the same. The same behavior is seen with losses. I would appreciate finding out what is going on.

import numpy as np
from tensorflow import keras
from tensorflow.keras import layers

num_classes = 10
input_shape = (28, 28, 1)

# the data, split between train and test sets
(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
print(x_train.shape,y_train.shape,x_test.shape,y_test.shape)
# Scale images to the [0, 1] range
x_train = x_train.astype(float32) / 255
x_test = x_test.astype(float32) / 255
# Make sure images have shape (28, 28, 1)
x_train = np.expand_dims(x_train, -1)  #adds another dimension at the end for color
x_test = np.expand_dims(x_test, -1)


# convert class vectors to binary class matrices
y_train = keras.utils.to_categorical(y_train, num_classes)
y_test = keras.utils.to_categorical(y_test, num_classes)

initializer = keras.initializers.RandomNormal(mean=0., stddev=1.)

model = keras.Sequential(
    [
        keras.Input(shape=input_shape),
        layers.Flatten(),
        layers.Dense(30, activation=sigmoid, use_bias=True,
                     kernel_initializer=initializer, 
                     bias_initializer=initializer),
        layers.Dense(num_classes, activation=sigmoid,
                     kernel_initializer=initializer, 
                     bias_initializer=initializer),    
    ]
)
batch_size = 10
epochs = 2
opt = keras.optimizers.SGD(learning_rate=3)
model.compile(loss=MeanSquaredError, optimizer=opt, metrics=[accuracy])
ann=model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_split=1/6)

# Evaluate the trained model
print(\n)
score = model.evaluate(x_train[:50000], y_train[:50000], verbose=0)
print(evaluate train accuracy:, score[1])
score = model.evaluate(x_train[-10000:], y_train[-10000:], verbose=0)
print(evaluate validation accuracy:, score[1])
print(history last train accuracy:,ann.history['accuracy'][-1])
print(history last validation accuracy:,ann.history['val_accuracy'][-1])

Topic keras

Category Data Science


training's loss/accuracy are not calculated for the whole dataset every time.
It is calculated for the current batch and averaged successively.
test's loss/accuracy is calculated at the end of an epoch for the whole test data.

Check these references
Keras FAQ
SO Answer

You may put the validation data same as the train data to check it.

ann=model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, validation_data=(x_train,y_train))

# Evaluate the trained model
print("\n")
score = model.evaluate(x_train, y_train, verbose=0)

enter image description here


Or, you may write a custom callback to print appropriate logs as suggested in the SO answer.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.