How Adaboost calculates error for each weak learner in training?
I am studying the Adaboost classification algorithm because i would like to implement it from scratch. I understand how it works, but i am not able to understand where some steps are placed.
I will describe the Adaboost training steps in my understanding (sorry for any incorrect formalism):
- Initialize a weak learner $k$
- Define a weight for each sample in the dataset equally $w =\frac{1}{N}$
- Fit $k$ to the dataset
- Calculate error $e = \sum_{i=0}^{N}e_iw_i$
- Calculate importance $\alpha$ of $k$, i.e. $\alpha = \frac{1}{2}log(\frac{1-e}{e})$
- Recalculate weights for the correct classified samples: $w_{t+1} = w e^{\alpha}$
- Recalculate weights for the incorrect classified samples: $w_{t+1} = w e^{-\alpha}$
- Normalize new sample weights: $w_{normalized} = \frac{w}{\sum_{i=0}^N w_i}$
- For all sequent learners, select samples based in weighted random choice until get a dataset with same size as the original and do the same proccess.
My question is: how error is obtained? Regarding the implementation, should i firstly fit the dataset and then get the error from predicting the same dataset? This doesn't seems correct.
I've tried to read different sources about this and even a great explanation from the Statquest channel wasn't able to make this clear.
Thanks!
Topic adaboost ensemble-modeling data-mining machine-learning
Category Data Science