How to specify scale_pos_weight value at runtime in Hyperopt?

Question

How to specify scale_pos_weight value at runtime in Hyperopt?

megjosh

2022年3月25日 06:06

I want to use LighgbmClassifier for a binary Classification. for Hyper Parameter tuning I want to use Hyperopt. The Dataset is imbalanced. Using Sklearns class_weight.compute_class_weight as shown below

        clas_wts_arr = class_weight.compute_class_weight('balanced',np.unique(y_trn),y_trn)   
        self.scale_pos_wt = clas_wts_arr[0] / clas_wts_arr[1]

The following is the space parameter that I am passing to the objective function

        space = {'objective' : hp.choice('objective', objective_list),
                 'boosting' : hp.choice('boosting', boosting_list),
                 'metric' : hp.choice('metric', metric_list),
                 max_depth: hp.quniform(max_depth, 1, 15,2),
                 'min_data_in_leaf': hp.quniform('min_data_in_leaf', 1, 256, 1),                     
                 'num_leaves': hp.quniform('num_leaves', 7, 150, 1),
                 'feature_fraction' : hp.quniform('feature_fraction', 0.5, 1, 0.01),
                 'min_gain_to_split' : hp.quniform('min_gain_to_split', 0.1, 5, 0.01),
                 'lambda_l1' : hp.uniform('lambda_l1', 0, 5),
                 'lambda_l2' : hp.uniform('lambda_l2', 0, 5),
                 'feature_pre_filter': False}

My question will the following set scale_pos_weight properly in the space dictionary

        #set scale pos weight explicitly
        space['scale_pos_weight'] = self.scale_pos_wt

If that is wrong then what would be the correct way to set scale_pos_weight at runtime in the space dictionary that is passed to the Objective fn that is in turn passed to the fmin of Hyperopt.

Thanks for your help and answers.

Topic lightgbm

Category Data Science

megjosh · Accepted Answer · 2020年10月24日 02:36

Use Sklearns class_weight.compute_class_weight, which will return an array of weights. My problem here is binary hence i had two elements in the array. We need to increase the weightage of the minority class. The target variable 1 was my minority class, thus I set the maximum scale_pos_weight so to the value from the second element in the array.

 clas_wts_arr = class_weight.compute_class_weight('balanced',np.unique(y_trn),y_trn)   
 self.scale_pos_wt = clas_wts_arr[1]

As part of the space dictionary passed to the objective fn defined within fmin we include scale_pos_weight like the following

'scale_pos_weight' :hp.choice('scale_pos_weight',scl_ps_wt)

if the fmin object has been defined as follows

            best = fmin(fn=objective,
                    space=space,
                    algo=tpe.suggest,
                    max_evals=self.NUM_EVALS, 
                    trials=trials)

Best will have the index of the best value in the array . Using this index we get the actual value of scale_pos_weight . We use this value to set the scale_pos_weight in best .

 best['scale_pos_weight'] = scl_ps_wt[best['scale_pos_weight']]

Now, use can set best as parameter to you classifier.

How to specify scale_pos_weight value at runtime in Hyperopt?

About