I am using three different of the shelf classifiers. It's a three class classification task. I want to calculate the optimal weights (c1weight, c2weight, c3weight) for each classifier (real task more classifiers and also weights for each class). Maybe simple grid search approach or sklearn ensemble classifier could do that. vc = VotingClassifier(estimators=[('gbc',GradientBoostingClassifier()), ('rf',RandomForestClassifier()),('svc',SVC(probability=True))], voting='soft',n_jobs=-1) params = {'weights':[[1,2,3],[2,1,3],[3,2,1]]} grid_Search = GridSearchCV(param_grid = params, estimator=vc) grid_Search.fit(X_new,y) print(grid_Search.best_Score_) I don't understand how to implement this for the following code. def get_classification(text, c1weight, …

I was wondering if there's a good way to use ensembling when I have two or more algoritims producing ranked lists. That is, suppose I have the following datasets consisting of ordered lists (higher to the top means more relevant): Method1_Rankings Method2_Rankings GoldStandard_Rankings item1 item2 item1 item3 item1 item3 item2 item10 item5 ... Is there a way to optimally combine methods 1 and 2 (e.g., give the rankings some weights or similar)? Thank you.

We are trying to replace an existing physical model (8 inputs/7 outputs) with artificial neural networks. The physics behind the existing model is mainly thermodynamics of humid air for air conditioning, with some turbomachinery involved, which yields most likely complex functions between inputs and outputs. One approach was already done: single output neural networks (10 NN with same # hidden layers but different parameters like batch size, # epochs, optimizer, etc). Then some sort of stacking ensembled was used: every …

I'm not quite sure how I should go about tuning xgboost before I use it as a meta-learner in ensemble learning. Should I include the prediction matrix (ie. df containing columns of prediction results from the various base learners) or should I just include the original features? I have tried both methods with just the 'n_estimators' tuned with F1 score as the metric for cross-validation. (learning rate =0.1) Method 1: With pred matrix + original features: n_estimators = 1 (this …

I tried to use ClusterEnsemble, getting error while using ClusterEnsemble package in Python: ERROR: Cannot install clusterensembles==0.2.7 and clusterensembles==1.0.0 because these package versions have conflicting dependencies. Is there any way I can use Voting or Bagging or Consensus matrix ensemble method for UnSupervised learning in Python? Or, Adaptive Clustering Ensemble? Please advise.

Is it possible to build ensemble models without a decision tree? I know that the description of ensembles itself suggests otherwise. However, I am really new to machine learning and all the ensemble models I came across so far use or are described using a decision tree.

Suppose I have two binary classifiers, A and B. Both are trained on the same set of data, and produce predictions on a different (but same for both classifiers) set of data. The precision for A is high and the recall is low, whereas the precision for B is low and the recall is high. Is there a way to combine these two models where I can get the precision from one and recall from the other, or possibly use …

I am trying a simple model ensemble with 2 different input datasets and 1 output. I want to get predictions for one dataset and hope model will extract some useful features from the second one. I get an error: ValueError: Data cardinality is ambiguous: x sizes: 502, 1002 y sizes: 502 Make sure all arrays contain the same number of samples. But I want it to fit for smaller dataset. Architecture is like this: Code: import pandas as pd from …

When I open the Gradient Boosting widget xgboost is grayed out even though it is described in detail under the documentation section on Orange. The two options are Gradient Boosting (scikit-learn) and catboost. Is there a way to activate or install xgboost?

I am trying to apply weighted majority voting on an ensemble as a combiner method. I read different papers and articles, however, I am still a bit lost on: How the weighted majority voting works How to assign a weight for every ensemble base classifiers when using weighted majority voting.

What are some good models to complement XGBOOST via stacking in typical Kaggle datascience competition? I realize XGBoost with well-tuned hyperparamters are generally quite good already.

I am working on a classification problem for predicting whether the shipment is going to be late or not. I would say the classifier is mediocre at predicting the positive class at the moment. But the ambition is to improve it. However, after doing some analysis, I have found out that there is an important component (Customs) that has appeared as the cause of shipment delay in majority of FN. Currently, I don't have a feature that would directly be …

I have trained two classifiers .. Text Classification and Image Classification. So both models gives score for each class. For example there are 3 classes. Each model give array of 3 confidence score for each class. I want to write some decider function to predict the final class based on these two model results. Dataset is very less around 100 observations. What possible solution i can try.

I was thinking if i can use polynomial regression like a weak learners in gradient boosting but i read that decision trees are used for that and i cannot find anything that show me the possibility of other weak learners could be used.

I am working on a binary classification problem with unbalanced data (17% for positive class). The problem is as following: My three individual models when predicting on the test set (for which I don't have the labels) gives quite similar distribution as for Train set. But ensemling these models, while giving slighltly better result (F1-score), it drastically changes the distribution on Test set going from ~20% to 5%. My question is : I am confused between choosing the best individual …

How does stacking help in terms of bias and variance? I have a hunch that stacking can help reduce bias but i am not sure, could someone refer to a paper?

This is a citation from "Hands-on machine learning with Scikit-Learn, Keras and TensorFlow" by Aurelien Geron: "Bootstrapping introduces a bit more diversity in the subsets that each predictor is trained on, so bagging ends up with a slightly higher bias than pasting, but this also means that predictors end up being less correlated so the ensemble’s variance is reduced." I can't understand why bagging, as compared to pasting, results in higher bias and lower variance. Can anyone provide an intuitive …

I have two models, $m_1$ and $m_2$, and I want to ensemble them into a final model. I want to be able weight one or the other more according to a grid search. There are two main ideas that come to my mind when doing so: Define a family of models $m_1 \cdot a + m_2 \cdot (1 - a)$, where $0 < a < 1$, find the $a$ that gives the best score. Define a family of models $m_1^a …

I got similar results in 2 models which consists of similar algorithms. Model 1 with cv=10 has a f1'micro' of 0.941. See code below. Model 2 only train test split (no cv) has f1'micro' 0.953. Now here is my understanding problem. Before I did a Grid-Search to find best hyperparameters. Now I would like to do just a cross validation to train the dataset. Like the red marked in the picture. In the code there is still the Grid Search …

I have a training and a test dataset. I would like to use the output of Model A in an ensemble model. However, I would like to use early stopping. Usually, I would create Model A for each K-fold (on training) and predict the OOF to create the meta-model dataset. Then I would repeat the methodology but for hyperparameter tuning the 2nd layer of the stacking model, trained with the meta-model dataset. Finally, I train the meta models on the …