xgboost performance
XGBoostRegressor
is not performing better than AdaBoostRegressor
for the same set of parameters for some reason. Since my dataset is big, I made an example using sklearn's make_regression
as follows.
from sklearn.ensemble import AdaBoostRegressor
from sklearn.datasets import make_regression
from sklearn.tree import DecisionTreeRegressor
from sklearn.linear_model import LinearRegression
from xgboost import XGBRegressor
X, y = make_regression(n_samples=10000, n_features=1, n_informative=1,
random_state=0, noise=1,shuffle=False)
regr = LinearRegression()
regr.fit(X, y)
print regr.score(X, y)
regr = AdaBoostRegressor(DecisionTreeRegressor(max_depth=6),n_estimators=10,random_state=0)
regr.fit(X, y)
print regr.score(X, y)
regr = XGBRegressor(max_depth=6,n_estimators=10,random_state=0)
regr.fit(X, y)
print regr.score(X, y)
Result
0.988885847388612
0.9891515914701008
0.8660647073587461
I want the same max_depth and n_estimators for comparison purposes. What, if any, is the hyper-parameter making the difference?
Topic bagging boosting xgboost
Category Data Science