Getting dummies for both train and test data

Question

Getting dummies for both train and test data

A Arbitrage

2022年3月26日 11:02

Should I apply pd.get_dummies() for both train and test data? And would it not result in data leakage?

Oxbowerce · Accepted Answer · 2022年1月8日 12:20

If you use pandas.get_dummies on the train and test data separately you will likely run into issues because it is likely that there are new values in the test dataset which are not in the training dataset. It is therefore better to use something like sklearn.preprocessing.OneHotEncoder which can save state and encode the test dataset based on the values that were seen in the training dataset.

Getting dummies for both train and test data

About