How to add the Luong Attention Mechanism into CNN?

As I write my CNN model for an image binary classification below, I'm trying to add an attention layer to this model. I read from tf.keras.layers.Attention: https://www.tensorflow.org/api_docs/python/tf/keras/layers/Attention

But I still don't know exactly how to use it, any help is appreciated.

model = keras.Sequential()

model.add(Conv2D(filters = 64, kernel_size = (3, 3), activation = 'relu', padding='same', input_shape = ((256,256,3))))
model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2)))

model.add(Conv2D(filters = 128, kernel_size = (3, 3), activation = 'relu', padding='same'))
model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2)))

model.add(Conv2D(filters = 256, kernel_size = (3, 3), activation = 'relu', padding='same'))
model.add(MaxPooling2D(pool_size = (2, 2), strides=(2, 2)))

model.add(Flatten())

model.add(Dense(units = 512, activation = 'relu'))
model.add(Dropout(rate = 0.5))

model.add(Dense(units = 1,activation='sigmoid'))

Topic attention-mechanism keras convolutional-neural-network

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.