> For the complete documentation index, see [llms.txt](https://hwasumok.gitbook.io/r-flow/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://hwasumok.gitbook.io/r-flow/python/machine-learning/untitled.md).

# Ensemble Methods

Adaboost, Bagging, ExtraTrees,  GradientBoosting, GradientBoosting, Isolation Forest, Random Forest, Stacking, Voting, HistGradientBoosting

## AdaBoost, Bagging

For both Classifier & Regressor

### Input Data

|                      | Inputs                                                                                   |
| -------------------- | ---------------------------------------------------------------------------------------- |
| **\*Target Data(X)** | Data Input                                                                               |
| **\*Model Name**     | Name of the user-defined model                                                           |
| **Predict Output**   | Name of the output data                                                                  |
| **Merge Data**       | **Predict Output** will be created with predicted labels concatenated to **Merge Data**. |
| **\*X**              | User-selected columns to be X.                                                           |
| **\*Y**              | User-selected columns to be Y.                                                           |

### Defining estimators

In case of using **AdaBoost and Bagging models**, you have to **define the base\_estimator** in **Arguments.**

[**About R-Flow estimators** ](https://hwasumok.gitbook.io/r-flow/python/machine-learning/estimator)

![\[Task Information of AdaBoost Classifier/Regressor\]](https://3929524962-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGaxNdSdRN_L_WNiX4I%2F-MKza7ZIxpRFdAoKVcNA%2F-MKzcnFpRSvL-85ebVEC%2Fimage.png?alt=media\&token=8caf1bb8-b090-4016-81a9-fffb4b04b5a7)

## ExtraTrees, GradientBoosting, RandomForest, HistGradientBoosting

For both Classifier & Regressor

### Input Data

|                    | Inputs                                                                                   |
| ------------------ | ---------------------------------------------------------------------------------------- |
| **\*Target Data**  | Data Input                                                                               |
| **\*Model Name**   | Name of the user-defined model                                                           |
| **Predict Output** | Name of the output data                                                                  |
| **Merge Data**     | **Predict Output** will be created with predicted labels concatenated to **Merge Data**. |
| **\*X**            | User-selected columns to be X.                                                           |
| **\*Y**            | User-selected columns to be Y.                                                           |

![\[Task Information of ExtraTreeClassifier/Regressor\]](https://3929524962-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGaxNdSdRN_L_WNiX4I%2F-MKza7ZIxpRFdAoKVcNA%2F-MKzdadoQx73ASbObACa%2Fimage.png?alt=media\&token=4bbe53fb-2387-4bfb-a9c0-1748af1cddf8)

## IsolationForest

### Input Data

|                    | Inputs                                                                                   |
| ------------------ | ---------------------------------------------------------------------------------------- |
| **\*Target Data**  | Data Input                                                                               |
| **\*Model Name**   | Name of the user-defined  model                                                          |
| **Predict Output** | Name of the output data                                                                  |
| **Merge Data**     | **Predict Output** will be created with predicted labels concatenated to **Merge Data**. |
| **\*X**            | User-selected columns to be X                                                            |

![\[Task Information of IsolationForest\]](https://3929524962-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGaxNdSdRN_L_WNiX4I%2F-MKza7ZIxpRFdAoKVcNA%2F-MKzfUam2V74Vy4-qJwf%2Fimage.png?alt=media\&token=824f2b3d-03e5-4fdf-81c1-6167ed1bb79d)

## Stacking, Voting

For both classifier & regressor

### Input Data

|                      | Inputs                                                                                   |
| -------------------- | ---------------------------------------------------------------------------------------- |
| **\*Target Data(X)** | Data Input                                                                               |
| **\*Model Name**     | Name of the user-defined model                                                           |
| **Predict Output**   | Name of the output data                                                                  |
| **Merge Data**       | **Predict Output** will be created with predicted labels concatenated to **Merge Data**. |
| **\*X**              | User-selected columns to be X.                                                           |
| **\*Y**              | User-selected column to be Y.                                                            |

![](https://3929524962-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGaxNdSdRN_L_WNiX4I%2F-MKza7ZIxpRFdAoKVcNA%2F-MKzhHHUuQUd6fc2y6nR%2Fimage.png?alt=media\&token=434da81d-7d19-42fb-80c3-efea9b6c4c60)

### Defining estimators

In case of using **Stacking & Voting models,** stack the  base\_estimators in Arguments.

Each element of the list is defined as a tuple of string (i.e. user-defined name of the estimator) and an estimator instance.

Define the 'estimators' in Argument as below: 

**\[('str1', estimator1),('str2', estimator2)]**    (ex. \[('rf', RF),('gdb',GDB),('adb',ADB)] )

![\[example of StackingClassifier estimators\]](https://3929524962-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGaxNdSdRN_L_WNiX4I%2F-MKza7ZIxpRFdAoKVcNA%2F-MKzjAaWQB6pj7beTS_q%2Fimage.png?alt=media\&token=89dabf7f-0861-455c-84a1-4a7e33b9fbbf)

## Example Workflow

[R-Flow Task Example Video- StackingClassifier](https://youtu.be/EOVTvM2OCKk)

![\[Example of Stacking Classifier\]](https://3929524962-files.gitbook.io/~/files/v0/b/gitbook-legacy-files/o/assets%2F-MGaxNdSdRN_L_WNiX4I%2F-MH5PZClOckUPNzn4N-M%2F-MH5QtFBFMJ6cn-sA_Pj%2Fimage.png?alt=media\&token=7b9b2dfe-7a63-4c68-8245-c616e1f874ba)

## Python Package

**AdaBoostClassifier:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostClassifier.html)

**AdaBoostRegressor:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostRegressor.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.AdaBoostRegressor.html)

**BaggingClassifier:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingClassifier.html)

**BaggingRegressor:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingRegressor.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.BaggingRegressor.html)

**ExtraTreesClassifier:** [**https://scikit-learn.org/0.15/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html**](https://scikit-learn.org/0.15/modules/generated/sklearn.ensemble.ExtraTreesClassifier.html)

**ExtraTreeRegressor:** [**https://scikit-learn.org/0.15/modules/generated/sklearn.ensemble.ExtraTreesRegressor.html**](https://scikit-learn.org/0.15/modules/generated/sklearn.ensemble.ExtraTreesRegressor.html)

**GradientBoostingClassifier:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingClassifier.html)

**GradientBoostingRegressor:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.GradientBoostingRegressor.html)

**IsolationForest:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.IsolationForest.html)

**RandomForestClassifier:** [**http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html**](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)

**RandomForestRegressor:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html)

**StackingClassifier:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingClassifier.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingClassifier.html)

**StackingRegressor:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingRegressor.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.StackingRegressor.html)

**VotingClassifier:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.VotingClassifier.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.VotingClassifier.html)

**VotingRegressor:** [**https://scikit-learn.org/0.22/modules/generated/sklearn.ensemble.VotingRegressor.html**](https://scikit-learn.org/0.22/modules/generated/sklearn.ensemble.VotingRegressor.html)

**HistGradientBoostingClassifier:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingClassifier.html)

**HistGradientBoostingRegressor:** [**https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html**](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.HistGradientBoostingRegressor.html)