Train an LSTM on separate sequences of different lengths
My case is the following: I want to train a sequential classifier to recognize what action is being performed given sensors observations.My data consists of 10 executions of an assembling task for 10 different people. So, basically each person performed the same task and I have the sensor measurements for each millisecond. That means that for each person I have a really big data set with the corresponding measurements and the labels (which action is being performed) for each millisecond. As a dataframe that looks like this:
millisecond sensor1 sensor2 sensor3 ... annotation
1 0.53 0.94 0.05 walk
2 0.57 0.84 0.15 walk
3 0.0 0.10 0.05 sit
............
So, I have 10 such dataframes in total, but each dataframe has a different length since each participant needed different time to complete the task. I am now wondering how to train an LSTM based on these independent sequences of actions. What I thought about:
a) maybe just merge several dataframes (let's say 9 dataframes) into one and use them as a training set and the other one as test.
b) again take 9 experiment runs as train and 1 as a test set. I then perform the training on each of the 9 sequences separately in an iterative way and after finishing training for a given sequence, I save the model, then load it again and continue training with the new sequence. In this case in Keras I will have : model.save - model.load -model.fit(new_sequence_dataframe).
c) merge dataframes again as in a), but introducing some BEGIN_EXPERIMENT and END_EXPERIMENT which I will concatenate at the begin/end of each sequence. The labels will indicate where the new sequence starts and they will have measurements 0 for all sensors.
How to proceed? Which procedure to use? Thank you!
the them in order to train an LSTM which can then for given sensor observations recognize/classify the action being performed. My thoughts: Merge 9 dataframes into one dataframe and train an LSTM with
Topic lstm keras tensorflow sequence dataset
Category Data Science