Why using the hyperbolic tangent or the sigmoid as activation function on the last layer gaves the same result in accuracy?

The problem

I'm making a simple Multilayer Perceptron (MLP), in Keras, that has to do the binary classification from some float type of data. Each single data is a group of three float values (e.g. 32.01, -10.23, -1.01) and is labelled with the value 0 or 1. Every time I do the training process the result of the validation accuracy and validation loss remain at the same value after few training epochs, like 5 or 6.

The problem is the validation accuracy don't increase and always remain at the value of 0.0000

What I have tried

I have tried to use a different activation function for the last layer, like a softmax, and while I got a different result I don't think doing this is 100% correct because the output I want from the MLP is only one, which is a value that should tell me whenever the data goes to a class or the another.

The source code

Here is the python code I use for making the model.

from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Sequential
from tensorflow.keras.activations import softmax, sigmoid
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import binary_crossentropy
from tensorflow.keras.metrics import categorical_accuracy

x_train = #training set data
y_train = #training set labels

# Make the model
model = Sequential()
model.add(Input(shape=(3,)))
model.add(Dense(8, activation=relu))
model.add(Dense(2, activation=tanh))

model.compile(optimizer=Adam(learning_rate=0.1),
              loss=binary_crossentropy,
              metrics=[binary_accuracy])

model.fit(x=x_train, 
          y=y_train,
          batch_size=32,
          epochs=100,
          validation_split=0.1,
          shuffle=True,
          verbose=1)

What I want

I don't know what am I doing wrong, but I simply want the model to have a better result in terms of validation accuracy and validation loss.

Topic activation-function keras tensorflow python machine-learning

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.