xgboost performance

XGBoostRegressor is not performing better than AdaBoostRegressor for the same set of parameters for some reason. Since my dataset is big, I made an example using sklearn's make_regression as follows.

from sklearn.ensemble import AdaBoostRegressor
from sklearn.datasets import make_regression
from sklearn.tree import DecisionTreeRegressor
from sklearn.linear_model import LinearRegression
from xgboost import XGBRegressor

X, y = make_regression(n_samples=10000, n_features=1, n_informative=1,
                       random_state=0, noise=1,shuffle=False)

regr = LinearRegression()
regr.fit(X, y)
print regr.score(X, y)

regr = AdaBoostRegressor(DecisionTreeRegressor(max_depth=6),n_estimators=10,random_state=0)
regr.fit(X, y)
print regr.score(X, y)

regr = XGBRegressor(max_depth=6,n_estimators=10,random_state=0)
regr.fit(X, y)
print regr.score(X, y)

Result

0.988885847388612
0.9891515914701008
0.8660647073587461

I want the same max_depth and n_estimators for comparison purposes. What, if any, is the hyper-parameter making the difference?

Topic bagging boosting xgboost

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.