How can I assess feature importance when determining whether a missing data is MCAR or not?

I was reading some lecture notes on missing data and the author suggests the following approach to determine whether some varibale is missing completely at random (MCAR) or not:

Supervised Learning method:

1. Code ‘missing’ as a new category.
2. Run a supervised analysis (to predict a separate target variable) and check if ‘missing’ has an effect on the prediction of the response in the learned model.
3. If the category ‘missing’ has an effect, this is evidence that data is not MCAR.

How can I assess whether the category 'missing' has an effect? What I know is that I can check the feature importance for say, Decision Tree model. But this gives me just a number. How can I decide based on that whether there is an effect there or not?

I know that, in regression, one can test whether there is a relation between variables using the $p$-value and the student distribution. This seems to be exactly the answer, but it is specific to Regression I suppose. (I can't see this being done for KNN or Neural Networks for instance).

Is there a more general way of assessing 'effect' in this case?