Q. Why is my testing accuracy varying slightly with batch size (97.7% - 100%)?

As you will see, when batch size is set to 1, I'm consistently getting 97.7% testing accuracy for all 10 iterations. However, when batch size is set to 64, I'm getting a testing accuracy of 100% 7 out of 10 times and 95% the remaining 3 times. So why is this happening? Something to do with Batch Norm? What batch size should I always set when testing a Transfer Learning model regardless of the size of the testing dataset(i.e., no of images in it)? I am a beginner in Transfer Learning applications and am using InceptionV3 with a custom classification head for binary classification on images, with the first 7 layers frozen, and the testing dataset had 1280 images(equally divided into 2 classes). Thank you.

# evaluate the performance of the model

for i in range(1, 11):
  evaluate = model.evaluate(test_generator, **steps = test_generator.n // 1**, verbose =1)
  print('Accuracy Test : {}'.format(evaluate[1]))

1280/1280 [==============================] - 30s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**

1280/1280 [==============================] - 28s 22ms/step - loss: 0.0867 - accuracy: 0.9766
Accuracy Test : **0.9765625**



# evaluate the performance of the model

for i in range(1, 11):
  evaluate = model.evaluate(test_generator, steps = **test_generator.n // 64**, verbose =1)
  print('Accuracy Test : {}'.format(evaluate[1]))

20/20 [==============================] - 3s 179ms/step - loss: 0.0012 - accuracy: 1.0000

Accuracy Test : **1.0**

20/20 [==============================] - 4s 190ms/step - loss: 0.0556 - accuracy: 1.0000

Accuracy Test : **1.0**

20/20 [==============================] - 5s 243ms/step - loss: 0.0235 - accuracy: 1.0000

Accuracy Test : **1.0**

20/20 [==============================] - 4s 223ms/step - loss: 0.1115 - accuracy: 0.9500

Accuracy Test : **0.949999988079071**

20/20 [==============================] - 4s 194ms/step - loss: 0.1605 - accuracy: 0.9500

Accuracy Test : **0.949999988079071**

20/20 [==============================] - 4s 197ms/step - loss: 0.4465 - accuracy: 0.9500

Accuracy Test : **0.949999988079071**

20/20 [==============================] - 4s 201ms/step - loss: 0.0363 - accuracy: 1.0000

Accuracy Test : **1.0**

20/20 [==============================] - 3s 134ms/step - loss: 7.0283e-04 - accuracy: 1.0000

Accuracy Test : **1.0**

20/20 [==============================] - 4s 183ms/step - loss: 0.0181 - accuracy: 1.0000

Accuracy Test : **1.0**

20/20 [==============================] - 3s 177ms/step - loss: 0.0123 - accuracy: 1.0000

Accuracy Test : **1.0**