How to train/test/validate hierachical classifiers?

I am writing an algorithm which allows to detect activities based on wearable data. I would like to try it out an hierachical approach (Local Classifier Per Parent Node structure). In the first level, I determine the intensity of the activity (1 classifier), and in the second level I determine the activity label (3 classifiers).

I am however struggling with how I need to approach the training/testing/validation of such a structure. What I did now is:

  • Split data into 2 parts: 75%/25%
  • First part of the data (75%) is used to train the 1st level classifier (intensity label) the 3 2nd level classifiers (activity label). For the 2nd level, I only trained the classifier with the samples that correspond with the parent node.
  • Second part of the data (25%) is then used as input for the first level classifier. The output data of the first level classifier is then used as input for the 2nd level classifiers, based on the label that was assigned in the 1st level.

I got the feeling that I am not properly testing and/or validating the 2nd level classifiers. Is there anyone who can give some guidance in how to approach this?

Thank you a lot in advance!

Topic activity-recognition training cross-validation hierarchical-data-format machine-learning

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.