Is Recursive Feature Elimination finding best features subset?

Question

Is Recursive Feature Elimination finding best features subset?

dzieciou

2020年9月21日 18:41

On a set of 9 features I have applied Recursive Feature Elimination (RFE) algorithm using SVM estimator, following approach from (1). When requesting a subset of size 1 to be found, then RFE returned feature X.

However, when I trained SVM over each feature individually, I found another feature Y to have higher accuracy than SVM trained over X.

I thought that RFE finds features with the highest accuracy.

Is my understanding of RFE wrong?

(1): Gene selection for cancer classification using support vector machines

Topic rfe svm feature-selection

Category Data Science

Ben Reiniger · Accepted Answer · 2020年9月21日 18:41

No, RFE cannot guarantee that it finds the feature subset with optimal score.

As with most greedy processes, the point of RFE is to reduce the computational cost (fitting a model for each of the $2^m$ feature subsets), at the cost of perhaps not finding the actual optimum (but hopefully "close enough").

See also https://stats.stackexchange.com/questions/232461/question-about-recursive-feature-elimination

Is Recursive Feature Elimination finding best features subset?

About