When i'm training the pre-trained model on kaggle dataset, it is showing 79% accuracy. when i test on different dataset it shows 42% accuracy
This is the code i used to train the pre-trained model 'inception v3'. And can i know it is showing accuracy as 0.8717 but the val_accuracy as 0.7781. why there is a that much difference. Can anyone explain where it is training correctly or not. If not, can anyone suggest me what wrong is happening?
Thanks in advance
below is the Code:
D:\NEW_DRD\Scripts\python.exe D:/NEW_DRD/own_model.py
2022-04-11 05:58:26.638080: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-04-11 05:58:26.639003: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Found 43360 validated image filenames belonging to 5 classes.
Found 4817 validated image filenames belonging to 5 classes.
Found 12045 validated image filenames belonging to 5 classes.
2022-04-11 05:59:49.643475: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2022-04-11 05:59:49.649688: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-04-11 05:59:49.661769: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: LAPTOP-GV2TL8N2
2022-04-11 05:59:49.662122: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: LAPTOP-GV2TL8N2
2022-04-11 05:59:49.691655: I tensorflow/core/platform/cpu_feature_guard.cc:151] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Epoch 1/80
2022-04-11 06:00:03.913777: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 194658304 exceeds 10% of free system memory.
1/678 [..............................] - ETA: 1:47:38 - loss: 1.8145 - accuracy: 0.1094 - auc: 0.34042022-04-11 06:00:06.382104: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 194658304 exceeds 10% of free system memory.
2/678 [..............................] - ETA: 18:05 - loss: 5.7475 - accuracy: 0.2578 - auc: 0.5568 2022-04-11 06:00:08.369249: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 194658304 exceeds 10% of free system memory.
3/678 [..............................] - ETA: 20:15 - loss: 4.8893 - accuracy: 0.3125 - auc: 0.61232022-04-11 06:00:10.411100: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 194658304 exceeds 10% of free system memory.
4/678 [..............................] - ETA: 21:08 - loss: 4.6196 - accuracy: 0.2617 - auc: 0.57742022-04-11 06:00:12.517928: W tensorflow/core/framework/cpu_allocator_impl.cc:82] Allocation of 194658304 exceeds 10% of free system memory.
678/678 [==============================] - 1888s 3s/step - loss: 1.2315 - accuracy: 0.5287 - auc: 0.8022 - val_loss: 1.1122 - val_accuracy: 0.5643 - val_auc: 0.8373
Epoch 2/80
678/678 [==============================] - 1926s 3s/step - loss: 1.0866 - accuracy: 0.5769 - auc: 0.8453 - val_loss: 1.0321 - val_accuracy: 0.5985 - val_auc: 0.8604
Epoch 3/80
678/678 [==============================] - 1943s 3s/step - loss: 1.0073 - accuracy: 0.6046 - auc: 0.8677 - val_loss: 0.9807 - val_accuracy: 0.6137 - val_auc: 0.8739
Epoch 4/80
678/678 [==============================] - 1951s 3s/step - loss: 0.9192 - accuracy: 0.6345 - auc: 0.8899 - val_loss: 0.9090 - val_accuracy: 0.6344 - val_auc: 0.8919
Epoch 5/80
678/678 [==============================] - 1940s 3s/step - loss: 0.8454 - accuracy: 0.6581 - auc: 0.9066 - val_loss: 0.8378 - val_accuracy: 0.6581 - val_auc: 0.9080
Epoch 6/80
678/678 [==============================] - 1955s 3s/step - loss: 0.7870 - accuracy: 0.6771 - auc: 0.9186 - val_loss: 0.8479 - val_accuracy: 0.6639 - val_auc: 0.9073
Epoch 7/80
678/678 [==============================] - 1961s 3s/step - loss: 0.7523 - accuracy: 0.6906 - auc: 0.9255 - val_loss: 0.7968 - val_accuracy: 0.6678 - val_auc: 0.9167
Epoch 8/80
678/678 [==============================] - 1951s 3s/step - loss: 0.7068 - accuracy: 0.7069 - auc: 0.9337 - val_loss: 0.7846 - val_accuracy: 0.6672 - val_auc: 0.9187
Epoch 9/80
678/678 [==============================] - 1986s 3s/step - loss: 0.6702 - accuracy: 0.7205 - auc: 0.9402 - val_loss: 0.7091 - val_accuracy: 0.6977 - val_auc: 0.9337
Epoch 10/80
678/678 [==============================] - 2096s 3s/step - loss: 0.6619 - accuracy: 0.7221 - auc: 0.9414 - val_loss: 0.7041 - val_accuracy: 0.7065 - val_auc: 0.9342
Epoch 11/80
678/678 [==============================] - 2002s 3s/step - loss: 0.6406 - accuracy: 0.7287 - auc: 0.9451 - val_loss: 0.7310 - val_accuracy: 0.6938 - val_auc: 0.9299
Epoch 12/80
678/678 [==============================] - 2018s 3s/step - loss: 0.6080 - accuracy: 0.7435 - auc: 0.9503 - val_loss: 0.6629 - val_accuracy: 0.7177 - val_auc: 0.9414
Epoch 13/80
678/678 [==============================] - 2033s 3s/step - loss: 0.5847 - accuracy: 0.7528 - auc: 0.9540 - val_loss: 0.6573 - val_accuracy: 0.7274 - val_auc: 0.9433
Epoch 14/80
678/678 [==============================] - 2029s 3s/step - loss: 0.5656 - accuracy: 0.7627 - auc: 0.9568 - val_loss: 0.6566 - val_accuracy: 0.7293 - val_auc: 0.9438
Epoch 15/80
678/678 [==============================] - 2341s 3s/step - loss: 0.5372 - accuracy: 0.7728 - auc: 0.9610 - val_loss: 0.6625 - val_accuracy: 0.7341 - val_auc: 0.9438
Epoch 16/80
678/678 [==============================] - 2184s 3s/step - loss: 0.5221 - accuracy: 0.7804 - auc: 0.9632 - val_loss: 0.6358 - val_accuracy: 0.7486 - val_auc: 0.9477
Epoch 17/80
678/678 [==============================] - 2253s 3s/step - loss: 0.5095 - accuracy: 0.7893 - auc: 0.9649 - val_loss: 0.6832 - val_accuracy: 0.7378 - val_auc: 0.9426
Epoch 18/80
678/678 [==============================] - 2413s 4s/step - loss: 0.4757 - accuracy: 0.8026 - auc: 0.9693 - val_loss: 0.7355 - val_accuracy: 0.7175 - val_auc: 0.9351
Epoch 19/80
678/678 [==============================] - 2538s 4s/step - loss: 0.4705 - accuracy: 0.8034 - auc: 0.9699 - val_loss: 0.6741 - val_accuracy: 0.7407 - val_auc: 0.9449
Epoch 20/80
678/678 [==============================] - 2148s 3s/step - loss: 0.4486 - accuracy: 0.8141 - auc: 0.9728 - val_loss: 0.6512 - val_accuracy: 0.7548 - val_auc: 0.9485
Epoch 21/80
678/678 [==============================] - 2108s 3s/step - loss: 0.4298 - accuracy: 0.8228 - auc: 0.9749 - val_loss: 0.6195 - val_accuracy: 0.7665 - val_auc: 0.9530
Epoch 22/80
678/678 [==============================] - 2145s 3s/step - loss: 0.4136 - accuracy: 0.8292 - auc: 0.9767 - val_loss: 0.6789 - val_accuracy: 0.7422 - val_auc: 0.9445
Epoch 23/80
678/678 [==============================] - 2108s 3s/step - loss: 0.3959 - accuracy: 0.8390 - auc: 0.9786 - val_loss: 0.7481 - val_accuracy: 0.7530 - val_auc: 0.9417
Epoch 24/80
678/678 [==============================] - 2121s 3s/step - loss: 0.3884 - accuracy: 0.8430 - auc: 0.9793 - val_loss: 0.6257 - val_accuracy: 0.7675 - val_auc: 0.9544
Epoch 25/80
678/678 [==============================] - 2115s 3s/step - loss: 0.3682 - accuracy: 0.8513 - auc: 0.9814 - val_loss: 0.7574 - val_accuracy: 0.7447 - val_auc: 0.9416
Epoch 26/80
678/678 [==============================] - 2102s 3s/step - loss: 0.3665 - accuracy: 0.8510 - auc: 0.9815 - val_loss: 0.5985 - val_accuracy: 0.7951 - val_auc: 0.9575
Epoch 27/80
678/678 [==============================] - 2125s 3s/step - loss: 0.3349 - accuracy: 0.8664 - auc: 0.9846 - val_loss: 0.6210 - val_accuracy: 0.7945 - val_auc: 0.9572
Epoch 28/80
678/678 [==============================] - 2107s 3s/step - loss: 0.3320 - accuracy: 0.8685 - auc: 0.9847 - val_loss: 0.6640 - val_accuracy: 0.7708 - val_auc: 0.9519
Epoch 29/80
678/678 [==============================] - 2114s 3s/step - loss: 0.3120 - accuracy: 0.8773 - auc: 0.9865 - val_loss: 0.6885 - val_accuracy: 0.7710 - val_auc: 0.9510
Epoch 30/80
678/678 [==============================] - 2163s 3s/step - loss: 0.3273 - accuracy: 0.8717 - auc: 0.9850 - val_loss: 0.6706 - val_accuracy: 0.7781 - val_auc: 0.9528
Epoch 31/80
678/678 [==============================] - 2365s 3s/step - loss: 0.2803 - accuracy: 0.8903 - auc: 0.9891 - val_loss: 0.6616 - val_accuracy: 0.7907 - val_auc: 0.9545
Epoch 32/80
678/678 [==============================] - 2200s 3s/step - loss: 0.2882 - accuracy: 0.8880 - auc: 0.9883 - val_loss: 0.6245 - val_accuracy: 0.7959 - val_auc: 0.9581
Epoch 33/80
678/678 [==============================] - 1951s 3s/step - loss: 0.2752 - accuracy: 0.8905 - auc: 0.9893 - val_loss: 0.6240 - val_accuracy: 0.8030 - val_auc: 0.9580
Epoch 34/80
678/678 [==============================] - 1960s 3s/step - loss: 0.2649 - accuracy: 0.8963 - auc: 0.9900 - val_loss: 0.6396 - val_accuracy: 0.8051 - val_auc: 0.9584
Epoch 35/80
678/678 [==============================] - 1969s 3s/step - loss: 0.2564 - accuracy: 0.9006 - auc: 0.9906 - val_loss: 0.7867 - val_accuracy: 0.7660 - val_auc: 0.9458
Epoch 36/80
678/678 [==============================] - 1941s 3s/step - loss: 0.2491 - accuracy: 0.9030 - auc: 0.9910 - val_loss: 0.6930 - val_accuracy: 0.8051 - val_auc: 0.9555
## Test Loss: 0.56536
## Accuracy on the test set: 79.70%
The first 5 predictions: ['Normal', 'Normal', 'Proliferate', 'Severe', 'Moderate']
{'Normal', 'Severe', 'Moderate', 'Mild', 'Proliferate'}
--------------
{'Normal', 'Severe', 'Moderate', 'Mild', 'Proliferate'}
Accuracy: 0.79701
Precision: 0.79215
Recall: 0.79701
F1 Score: 0.78612
Cohen Kappa Score: 0.71613
precision recall f1-score support
Mild 0.71 0.64 0.67 1721
Moderate 0.70 0.39 0.50 1721
Normal 0.74 0.86 0.79 5162
Proliferate 0.97 0.99 0.98 1721
Severe 0.95 0.99 0.97 1720
accuracy 0.80 12045
macro avg 0.81 0.77 0.78 12045
weighted avg 0.79 0.80 0.79 12045
Process finished with exit code 0
But when i tested on different dataset it is showing different accuracy. Can anyone explain me why this happened. and can anyone help how to overcome this. Thanks in advance.
## Test Loss NORM: 6.54364
## Accuracy on the test set NORM: 40.67%
The first 5 predictions NORM: ['Normal', 'Normal', 'Moderate', 'Normal', 'Normal']
{'Proliferate', 'Mild', 'Normal', 'Moderate', 'Severe'}
---------------------
{'Proliferate', 'Mild', 'Normal', 'Moderate', 'Severe'}
---------------------
set()
Accuracy NORM: 0.41492
Precision NORM: 0.35084
Recall NORM: 0.41492
F1 Score NORM: 0.35728
Cohen Kappa Score NORM: 0.03682
precision recall f1-score support
Mild 0.17 0.13 0.15 111
Moderate 0.26 0.08 0.12 300
Normal 0.51 0.75 0.61 542
Proliferate 0.05 0.02 0.03 88
Severe 0.10 0.17 0.13 58
accuracy 0.41 1099
macro avg 0.22 0.23 0.21 1099
weighted avg 0.35 0.41 0.36 1099
Topic opencv inception python machine-learning
Category Data Science