Why there is a marked difference in metric scores using linear regression or MLP as readout for echo state network?
I am using a reservoir computing architecture comprising of an echo state network as per the paper Reservoir Computing Approaches for Representation and Classification of Multivariate Time Series
Briefly, the architecture has four parts;
- Reservoir module (echo state network)
- Dimensional reduction module
- Representation module
- Readout module (linear regression, SVM or MLP)
For a multivariate time series classification task that I am doing, keeping all parameters the same in parts 1-3 from above, when I use linear regression as readout, I get an F1 score of about 0.25 and AUROC of about 0.58. When I use MLP as readout, I get an F1 score of about 0.4 (+0.15 from linear regression) and AUROC of about 0.8 (+0.22 from linear regression).
Quoting from the paper A Practical Guide to Applying Echo State Networks, section 3.1.;
For classification tasks, input data u(n) which are not linearly separable in the original space RNu, often become so in the expanded space RNx of x(n), where they are separated by Wou
My hypothesis for the difference in score between linear regression and MLP readout is that the output of the echo state network is not linearly separable and that's why MLP performs better as it is able to learn more complex patterns in the output. Is this correct?
If my hypothesis is correct, what could be done to make the echo state network output more linearly separable to improve the performance when using linear regression?
I would like to use linear readout as it has a shorter training time than MLP and from what I understood from the literature, it is more common to use linear readout when using echo state networks.
Topic mlp linear-regression
Category Data Science