LSTM evaluation metric MAE explanation

I have a hard time understanding the LSTM model performance as I summarize my model as follow:

X_train.shape
(120, 7, 11)
y_train.shape
(120,)
X_test.shape
(16, 7, 11)
y_test.shape
 (16,)

model = keras.Sequential()
model.add(keras.layers.LSTM(100, input_shape=(X_train.shape[1], X_train.shape[2]), return_sequences = True))
model.add(keras.layers.Dropout(rate = 0.2))
model.add(keras.layers.LSTM(20))
model.add(keras.layers.Dropout(rate = 0.2))          
model.add(keras.layers.Dense(1))
model.compile(loss='mean_squared_error', optimizer=keras.optimizers.Adam(0.001), metrics = ['mae'])

history = model.fit(
X_train, y_train, 
epochs=60,
batch_size=5,
verbose= 0,
validation_split = 0.1,
shuffle=False
)

Based on the below plots, both MSE and MAE decrease in the training process and their corresponding values are near zero.

However the prediction is not precise enough as I realize:

y_pred = model.predict(X_test)
model.evaluate(X_test,y_test)
[0.04673878103494644, 0.15574690699577332]

So my question is, how does really my model perform? I mean how can interpret its performance,since both MSE and MAE seem to be low but the prediction values are not quite convincing.

Topic lstm performance

Category Data Science


You are getting loss near to 0 but, Your true distribution of y in the range of 0-1 so, that 0.04 loss may be high loss. Just get random model and check the loss. You will get to know how much you decreased the loss. I will suggest to use r^2metric for evaluation.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.