Modeling Encoder-Decoder according to instructions from a paper

I am new to this field and I was reading a paper Predicting citation counts based on deep neural network learning techniques. There the authors describe the code that they implemented if someone wants to reproduce the results. I tried to do this but I am not sure if I succeeded.

Here is their description:

-RNN module - SimpleRNN
-Output dimension of the encoder - 512
-The output layer - Dense layer
-Activation function - ReLU
-Overfitting prevention technique - Dropout with 0.2 rate
-Epochs - 100
Optimization algorithm - RMSProp
Learning rate - 10^{-5}
Batch size - 256

And here is my implementation. I am not sure if the model I created is sequence to sequence.

epocsh = 100
batch_size = 256
optimizer = keras.optimizers.RMSprop(lr=0.00001)
model =  keras.models.Sequential([
    keras.layers.SimpleRNN(512, input_shape=[X_train.shape[0], X_train.shape[1]],
                           activation='relu', return_sequences=True, dropout=0.2),
    keras.layers.Dense(9)
])

model.compile(loss='mse', optimizer=optimizer, metrics=[keras.metrics.RootMeanSquaredError()])

The summary of this model is:

_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn (SimpleRNN)       (None, 154521, 512)       266240    
_________________________________________________________________
dense (Dense)                (None, 154521, 9)         4617      
=================================================================
Total params: 270,857
Trainable params: 270,857
Non-trainable params: 0
_________________________________________________________________

Any help would be appreciated. Thank you!

Update: Here is the dataset.

year  venue  c1  c2  c3  c4  c5  c6  c7  c8  c9  c10  c11  c12  c13  c14
1989    234   0   1   2   3   4   5   5   5   5    8    8   10   11   12
1989    251   0   0   0   0   0   0   0   0   0    0    0    0    0    0
1990    346   0   0   0   0   0   0   0   0   0    0    0    0    0    0

I need to give as an input all the columns until c5, and try to predict the other c's (which are citation count for the upcoming years).

Topic keras rnn python

Category Data Science


What is the input and output? Assuming your input is a sequence of documents, you would need to convert it into some sort of embeddings (TF-IDF, any other). After that comes the RNN - which looks ok to me. The output of the RNN is equal to the number of timesteps (num of words in each doc?). For each timestep we predict using a Dense final layer. You have 9 outputs. Is it some sort of a classification across 9 different types of categories or levels of citation counts?

Overall looks ok with the above assumptions other than the fact that you are feeding raw data instead of using an embedding layer

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.