Autoencoders for the compression of time series

I am trying to use autoencoder (simple, convolutional, LSTM) to compress time series.

Here are the models I tried.

Simple autoencoder:

    from keras.layers import Input, Dense
    from keras.models import Model
    import keras

    # this is the size of our encoded representations
    encoding_dim = 50

    # this is our input placeholder
    input_ts = Input(shape=(2100,))
    # "encoded" is the encoded representation of the input
    encoded = Dense(encoding_dim, activation='relu')(input_ts) #, activity_regularizer=regularizers.l2(10e-5)
    # "decoded" is the lossy reconstruction of the input
    decoded = Dense(2100, activation='tanh')(encoded)

    # this model maps an input to its reconstruction
    autoencoder = Model(input_ts, decoded)

    # this model maps an input to its encoded representation
    encoder = Model(input_ts, encoded)

    # create a placeholder for an encoded (%encoding_dim%-dimensional) input
    encoded_input = Input(shape=(encoding_dim,))
    # retrieve the last layer of the autoencoder model
    decoder_layer = autoencoder.layers[-1]
    # create the decoder model
    decoder = Model(encoded_input, decoder_layer(encoded_input))

    autoencoder.summary()

    adamax = keras.optimizers.Adamax(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.01)
    autoencoder.compile(optimizer=adamax, loss='mean_absolute_percentage_error')

Convolutional autoencoder:

    from keras.layers import Input, Dense, Conv1D, MaxPooling1D, UpSampling1D
    from keras.models import Model

    window_length = 518

    input_ts = Input(shape=(window_length,1))

    x = Conv1D(32, 3, activation="relu", padding="valid")(input_ts)
    x = MaxPooling1D(2, padding="valid")(x)

    x = Conv1D(1, 3, activation="relu", padding="valid")(x)

    encoded = MaxPooling1D(2, padding="valid")(x)

    encoder = Model(input_ts, encoded)

    x = Conv1D(16, 3, activation="relu", padding="valid")(encoded)
    x = UpSampling1D(2)(x) 

    x = Conv1D(32, 3, activation='relu', padding="valid")(x)
    x = UpSampling1D(2)(x)

    decoded = Conv1D(1, 1, activation='tanh', padding='valid')(x)

    convolutional_autoencoder = Model(input_ts, decoded)

    convolutional_autoencoder.summary()

    optimizer = "nadam"
    loss = "mean_absolute_error"

    convolutional_autoencoder.compile(optimizer=optimizer, loss=loss)

LSTM autoencoder:

    from keras.layers import Input, LSTM, RepeatVector
    from keras.models import Model

    inputs = Input(shape=(1, 500))
    encoded = LSTM(128)(inputs)

    decoded = RepeatVector(1)(encoded)

    decoded = LSTM(500, return_sequences=True)(decoded)

    sequence_autoencoder = Model(inputs, decoded)
    encoder = Model(inputs, encoded)

    sequence_autoencoder.summary()

    sequence_autoencoder.compile(optimizer='nadam', loss='mean_absolute_error')

To check for compression loss, I use the SMAPE formula.

I got such results. The average loss for simple autoencoder is 14.28%, for convolutional autoencoder is 8.04%, for LSTM-autoencoder is 9.25%.

My question is: is it practical to compress time series with losses using a neural network if the compression time does not matter? Perhaps i should pay attention to other methods? How can I achieve better compression?

Thanks!

Topic lstm keras autoencoder convolution time-series

Category Data Science


Above all, you should take care of the time series. Despite from that, AEs are thoroughly used for time series, especially LSTM+AE. Did you vary the topology? For the CAE it looks reasonable but the other models lack some layers, or? Furthermore, some regular advices would be to standardize the input, change the activation functions (tanh worked well for me in the output layer) as well as the number of neurons per layer and the amount of layers in general. Could you provide the head() of the input data? An AE expects to fit X on X, maybe you missed that?

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.