Loading saved model fails

I've trained a model and saved it in .h5 format. when I try loading it I received this error ValueError Traceback (most recent call last) ~\AppData\Local\Temp/ipykernel_588/661726548.py in <module> 9 # returns a compiled model 10 # identical to the previous one ---> 11 reconstructed_model = keras.models.load_model("./custom_model.h5") ~\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py in error_handler(*args, **kwargs) 65 except Exception as e: # pylint: disable=broad-except 66 filtered_tb = _process_traceback_frames(e.__traceback__) ---> 67 raise e.with_traceback(filtered_tb) from None 68 finally: 69 del filtered_tb ~\Anaconda3\lib\site-packages\keras\utils\generic_utils.py in class_and_config_for_serialized_keras_object(config, module_objects, custom_objects, printable_module_name) 560 …
Category: Data Science

How to add the Luong Attention Mechanism into CNN?

As I write my CNN model for an image binary classification below, I'm trying to add an attention layer to this model. I read from tf.keras.layers.Attention: https://www.tensorflow.org/api_docs/python/tf/keras/layers/Attention But I still don't know exactly how to use it, any help is appreciated. model = keras.Sequential() model.add(Conv2D(filters = 64, kernel_size = (3, 3), activation = 'relu', padding='same', input_shape = ((256,256,3)))) model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2))) model.add(Conv2D(filters = 128, kernel_size = (3, 3), activation = 'relu', padding='same')) model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, …
Category: Data Science

I have a folder with 25000 images with cats and dogs picture - how to use tf.keras.utils.image_dataset_from_directory?

I am using vats-vs-dogs data set where the subdirectories are not precreated. Only 1 folder with 25000 images. (cat.001.jpg.... etc then dog.342.jpg.... etc) all in one folder. There are no separate folders called /cats or /dogs. How to use tf.keras.utils.image_dataset_from_directory function in such case ? Do I need to explicitly compartmentalize all dogs in /dog and all cats into /cats and then do it ? img_height = 300 img_width = 300 training_dir = 'drive/MyDrive/training1/dog-cat-train/train' train_ds = tf.keras.utils.image_dataset_from_directory( training_dir, labels='inferred', label_mode='categorical', …
Category: Data Science

Variational AutoEncoder giving negative loss

I'm learning about variational autoencoders and I've implemented a simple example in keras, model summary below. I've copied the loss function from one of Francois Chollet's blog posts and I'm getting really really negative losses. What am I missing here? Model: "model_1" __________________________________________________________________________________________________ Layer (type) Output Shape Param # Connected to ================================================================================================== input_1 (InputLayer) [(None, 224)] 0 __________________________________________________________________________________________________ encoding_flatten (Flatten) (None, 224) 0 input_1[0][0] __________________________________________________________________________________________________ encoding_layer_2 (Dense) (None, 256) 57600 encoding_flatten[0][0] __________________________________________________________________________________________________ encoding_layer_3 (Dense) (None, 128) 32896 encoding_layer_2[0][0] __________________________________________________________________________________________________ encoding_layer_4 …
Category: Data Science

False positive in Multi class Image classification

I am training a neural network with some convolution layers for multi class image classification. I am using keras to build and train the model. I am using 1600 images for all categories for training. I have used softmax as final layer activation function. The model predicts well on all True categories with high softmax probability. But when I test model on new or unknown data, it predicts with high softmax probability. How can I reduce that? Should I make …
Category: Data Science

Keras model with 3 input images giving wrong output

I have created a keras model that takes 3 images as input, passes them to individual CNN backbone(mobilenet_v2) and fuse the results from 3 individual streams. These fused outputs further goes through a FCN and gives probability values for 10 classes. Now when i pass 3 images to my model using model.predict(), I am getting an output of 3x10 (list of 3 outputs with 10 values in each). Here is the network snapshot and here is the output *[[0.04718336 0.07464679 …
Category: Data Science

Problems to understand how to create the input data for time series forecasting with a recurrent neural network in Keras

I just started to use recurrent neural networks (RNN) with Keras for time-series forecasting and I found this tutorial Forecasting with RNN. I have difficulties understanding how to build the training data both regarding the syntax and the format of the input data. Here is the code: import pandas as pd import numpy as np import tensorflow as tf from tensorflow import keras from matplotlib import pyplot as plt # Read the data for the parameters from a csv file …
Category: Data Science

When using padding in sequence models, is Keras validation accuracy valid/ reliable?

I have a group of non zero sequences with different lengths and I am using Keras LSTM to model these sequences. I use Keras Tokenizer to tokenize (tokens start from 1). In order to make sequences have the same lengths, I use padding. An example of padding: # [0,0,0,0,0,10,3] # [0,0,0,0,10,3,4] # [0,0,0,10,3,4,5] # [10,3,4,5,6,9,8] In order to evaluate if the model is able to generalize, I use a validation set with 70/30 ratio. In the end of each epoch …
Category: Data Science

How to make predictions of multiple input samples at once in tf 2 with keras

I am quite confused on the output of model.predict when after training I validate my model on around 6000 samples I use the following pseudo code: model.fit(...) predictions = model.predict(val_set) len(predictions) # == len(val_set) result: tensor array of shape=(len(tensor_array),14) (one prediction for each input sample) in production I currently use the following code to ocr image numbers: model = tf.keras.models.load_model('number_ocr_v2') def predict(list_images): global model print("length:") print(len(list_images)) #predictions = model.predict(list_images) # <- Same result predictions = model.predict_on_batch(list_images) print(len(predictions)) print(predictions) console Output: …
Category: Data Science

Training Inception V3 based model using Keras with Tensorflow Backend

I am currently training a few custom models that require about 12Gb GPU memory at the most. My setup has about 96Gb of GPU memory and python/Jupyter still manages to hog up all the gpu memory to the point that I get the Resource exhausted error thrown at me. I am stuck at this peculiar issue for a while and hence any help will be appreciated. Now, when loading a vgg based model similar to this: from keras.applications.vgg16 import VGG16 …
Category: Data Science

Strategy for achieving a “hybrid” GAN

I have worked with a few GAN-like algorithms, but always with similar inputs and outputs. Being only a novice in deep learning, I often work by adapting an already existing Notebook, but today I have a more complex problem and I cannot find a similar example. For my algorithm I need to predict a curve, which is a list of floats (e.g. dimension 20x1), from two types of input data; a 2D image (e.g. a 16x16 normalized float array) and …
Category: Data Science

How to fit a model on validation_data?

can you help me understand this better? I need to detect anomalies so I am trying to fit an lstm model using validation_data but the losses does not converge. Do they really need to converge? Does the validation data should resemble train or test data or inbetween? Also, which value should be lower, loss or val_loss ? Thankyou!
Category: Data Science

ValueError: Tensor Tensor("activation_5/Softmax:0", shape=(?, 2), dtype=float32) is not an element of this graph

There seem to be an issue with predicting using my keras model. I had trained it using the following keras code: model = Sequential() model.add(Conv2D(32, (3, 3), input_shape=(150, 150,3),padding='same')) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2),padding='same')) model.add(Conv2D(32, (3, 3),padding='same')) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2),padding='same')) model.add(Conv2D(64, (3, 3),padding='same')) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2, 2),padding='same')) model.add(Flatten()) # this converts our 3D feature maps to 1D feature vectors model.add(Dense(64)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(2)) model.add(Activation('softmax')) model.compile(loss='binary_crossentropy', optimizer='rmsprop', metrics=['accuracy']) However when i predict it on my local system after training with the shape (1,150,150,3) . …
Category: Data Science

Loss function to prevent estimator bias

I have a regression problem I'm trying to build a model for: Predicting sales per person (>= 0) depending on some variables. I'm running different model types and gave deep neural networks a try. The loss functions I'm using are mean squared error and mean absolute error (or sometimes a mix). I often run into this issue though, that despite mse and mae are being optimized, I end up with a very strong bias in the prediction, e.g. sum(training_all_predictions) / …
Category: Data Science

Strange Behavior for trying to Predict Tennis Millionaires with Keras (Validation Accuracy)

I'm trying to make an NN with Keras to predict the ATP players that will get more than US$1 million in prize money based on their weight and height (from a dataset I mined some weeks ago), but I have been getting a weird behavior especially for the validation accuracy. Sometimes it gets to 84-85%, which is reasonable since SVMs and GaussianNB seem to be able to hit only 83.3% at best (check this post for more info), but sometimes …
Category: Data Science

Binary Classification Comparing two time series of variable length

Is there a machine learning model (something like LSTM or 1D-CNN) that takes two time series of variable length as input and outputs a binary classification (True/False whether time series are of same label)? So the data would look something like the following date value label 2020-01-01 2 0 # first input time series 2020-01-02 1 0 # first input time series 2020-01-03 1 0 # first input time series 2020-01-01 3 1 # second input time series 2020-01-03 1 …
Category: Data Science

Keras - add_weight() method not adding to total model parameters

I am creating a custom Keras layer FConv2D(), and adding a weight in its build() function using the add_weight() method as suggested in official Keras tutorial for creating custom layers. def build(self, input_shape): shape = tf.TensorShape(input_shape).as_list() h = shape[1] w = shape[2] in_channels = shape[3] self.kernel = self.add_weight( shape=(h,w,in_channels,self.num_outputs), initializer="random_normal", trainable=True, ) super(FConv2D, self).build(input_shape) But when I print the summary of a single-layer model containing just this layer, the number of parameters in this layer is shown to be 0. …
Category: Data Science

Seems like the Keras .fit and .evaluate methods give different training accuracy (yet the same validation accuracy). Same thing on loss.?

The code below is a translation of Nielsen's first mnist code to Keras. Surprisingly, the last accuracy value of the .fit method and the accuracy value for the .evaluate method are different for the training data. As I expected, for the validation data the accuracies are the same. The same behavior is seen with losses. I would appreciate finding out what is going on. import numpy as np from tensorflow import keras from tensorflow.keras import layers num_classes = 10 input_shape …
Topic: keras
Category: Data Science

Negative loss values for adaptive loss in tensorflow

I have used adaptive loss implementation on a neural network, however after training a model long enough, I am getting negative loss values. Any help/suggestion would be highly appreciated! Please let me know if you need additional info Model definition - hyperparameter_space = {"gru_up": 64, "up_dropout": 0.2, "learning_rate": 0.004} def many_to_one_model(params): input_1 = tf.keras.Input(shape = (1, 53), name = 'input_1') input_2 = tf.keras.Input(shape = (1, 19), name = 'input_2') input_3 = tf.keras.Input(shape = (1, 130), name = 'input_3') input_3_flatten = …
Category: Data Science

Steps taking too long to complete

I'm trying to train a model which in my opinion is taking too long compared to other datasets given that it's taking about 9s to complete a step. I think that the problem is because the dataset is not being stored on ram, but I'm not sure of this. The code is the following: def load_data(): train_datagen = ImageDataGenerator(rescale=1./255, shear_range=0.2, zoom_range=0.2, horizontal_flip=True) train_generator = train_datagen.flow_from_directory(path1, target_size=(200, 200), batch_size=32, class_mode="binary") test_datagen = ImageDataGenerator(rescale=1./255) test_generator = test_datagen.flow_from_directory(path2, target_size=(200, 200),batch_size=32, class_mode="binary") return train_generator, …
Category: Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.