Can I apply feature selection before splitting by requiring selection occurs > 90% of time
I want to move the feature selection step to before splitting to save time and allow bigger input dataset. If, in repeated subsamples, a feature is selected in over X percentage of cases I will keep it. Alternatively use very low X to remove features that will clearly never be selected. I have read warnings against doing this including on this forum because of information leakage. Feature selection: Information leaking if done before CV-split? But if the feature would have been selected in almost all post split cases then where is the problem? Edit: it does involve the target features.
Topic data-leakage feature-selection
Category Data Science