Emotion Recognition with Multi-task Learning

Introduction

I am a beginner in Data Science and currently working on a learning project aimed at emotion recognition from a bio-medical sensor dataset.

The dataset consists of 8 sensors data from 20 subjects, here I have attached a screenshot of a very small part of the dataset to give you better insight into it:

So, as you can see, the Dataset is about:

  • Multiple subjects and for each subject, there are 11 columns of data.
  • Columns 1-8 are raw sensor data (EEG, EOG, GSR, Skin Temperature, IR Response, ... ) at 250 Hz.
  • Columns 9-11 are the corresponding labels.

What I want to do?

I want to use a Multi-task Learning to generate the output of three emotion levels - Arousal, Dominance Valance.

I did lots of research about it but unfortunately didn't find any clear guidance on what I should do. I found something related to VAD model, but not certain how to implement it for this dataset.

So, if someone could give me a guideline or path (the main steps that I should do) to work on it, I can search about it and also try to solve it step by step.

Topic machine-learning-model multitask-learning machine-learning

Category Data Science

1
answered at

I suppose your question is .... "How to do multi-task learning?"

multi-task diagram

Should you take a neural network approach, for instance: A simple example with the above diagram is to pass Columns 1-8 into the "Shared Layers".

The task specific layers would each be related to your three targets ie. Task 1 - Arousal, Task 2 - Dominance &, Task 3 - Valance.

Do have a look at this review article on multi-task learning and its various types. A more accessible article with some code examples here.

Addendum

Edit: To address OP's inquiry about how to implement the above. I am not sure of the tools you plan to use, I am familiar with Python mainly. The below is a guide or suggested approach for implementing the multi-task learning with a deep learning framework:

  1. Install Python, and get the keras and TensorFlow libraries, and other data science libraries (ie. pandas)
  2. Try implement a smaller system first, or a toy model (see here for example code -- just try making sure you can run a simple graph first)
  3. Try the a more complicated graph model
  4. Adapt the final code in the above article to have 3 loss functions, (ie. one each for valence, arousal and dominance)
  5. Use a subset of your data for the model -- making sure it runs. You want your input features as columns 1-8 and your three targets or outcomes as valence, arousal and dmomiance.
  6. Once model and code is running well, do a train-test split of your data (a 80:20 split, ie. 20% of the data is reserved as an unseen data). You can use sklearn test-train-split for this purpose.
  7. Train model on test data.
  8. Check your model by doing a prediction on the test data, and seeing how different (you can use metrics like absolute error or mean squared error) to measure this.
  9. To improve your model further, tweak around the hyperparameters - ie. changing the optimiser or learning rate.

This is a very broad overview, sorry I can't go into the specifics as this is a large topic. If you don't understand any of the above terms you can do a quick search on them and you should be on your way.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.