How can I use a validation set to tune the hyperparameters of an XGBClassifier?

I'm currently building a ranking model using an XGBClassifier.

I have training, testing, and validation sets. I want to use the validation set to tune the hyperparameters of the XGBClassifier before using it to train a model. Is this the right idea?

Currently, I'm getting a 56% accuracy score with the default XGBClassifier (or 51% accuracy if I run PCA before training the model, which greatly reduces the time it takes to train). If I try tuning the hyperparameters before training, the results do not get any better at all. Here is what I'm doing:

First, I try tuning the hyperparameters with a RandomizedSearchCV. The validation features and class labels have been gathered before this code is executed.

params = {
    learning_rate   : [0.05, 0.10, 0.15, 0.20, 0.30],
    max_depth       : [3, 4, 5, 6, 8, 10, 12, 15],
    min_child_weight: [1, 3, 5, 7],
    gamma           : [0.0, 0.1, 0.2, 0.3, 0.4],
    colsample_bytree: [0.3, 0.4, 0.5, 0.7]
}
classifier = XGBClassifier()
random_search = RandomizedSearchCV(classifier, param_distributions=params, n_iter=5, n_jobs=-1, cv=5, verbose=3)
scaling = MinMaxScaler(feature_range=(-1, 1)).fit(validation_features)
validation_features = scaling.transform(validation_features)
random_search.fit(validation_features, validation_labels)
print(random_search.best_estimator_)

I then take the parameters shown by the previous print statement and plug them into my XGBClassifier, like so:

classifier = XGBClassifier(base_score=0.5, booster='gbtree', colsample_bylevel=1,
              colsample_bynode=1, colsample_bytree=0.7, gamma=0.4,
              learning_rate=0.15, max_delta_step=0, max_depth=5,
              min_child_weight=3, missing=None, n_estimators=100, n_jobs=1,
              nthread=None, objective='multi:softprob', random_state=0,
              reg_alpha=0, reg_lambda=1, scale_pos_weight=1, seed=None,
              silent=None, subsample=1, verbosity=1)

pca = decomposition.PCA(n_components=3)
pca.fit(train1_features)
train1_features = pca.transform(train1_features)
test1_features = pca.transform(test1_features)
scaling = MinMaxScaler(feature_range=(-1, 1)).fit(train1_features)
train1_features = scaling.transform(train1_features)
test1_features = scaling.transform(test1_features)
score = get_score(classifier, train1_features, test1_features, train1_labels, test1_labels)

Is there something glaringly obvious I'm doing wrong here? I'm new to machine-learning. I've mainly been following ideas I've seen online. Should tuning the hyperparameters always show better results, or are there cases where it does not help? Any advice is appreciated!