{"id":5400,"date":"2020-09-07T17:06:17","date_gmt":"2020-09-07T17:06:17","guid":{"rendered":"https:\/\/dataaspirant.com\/?p=5400"},"modified":"2023-10-03T12:51:52","modified_gmt":"2023-10-03T07:21:52","slug":"ensemble-methods-bagging-vs-boosting-difference","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/ensemble-methods-bagging-vs-boosting-difference\/","title":{"rendered":"Ensemble Methods | Bagging Vs Boosting Difference"},"content":{"rendered":"
<\/div>

\"Bagging

Difference Between Bagging & Boosting Ensemble Methods<\/h1>\n

In the world of machine learning, ensemble learning methods are the most popular topics to learn. These ensemble methods have been known as the winner<\/strong>algorithms<\/strong>. In the data science competitions platform like Kaggle<\/strong><\/a>, machinehack, HackerEarth<\/strong> ensemble methods are getting hype as the top-ranking people in the leaderboard are frequently using these methods like bagging methods and boosting methods.<\/p>\n

For the data scientist roles, in interviews the difference<\/strong> between bagging and boosting most frequently asked question. <\/p>\n

So in this article, we are going to learn different kinds of ensemble methods. In particular, we are going to focus more on Bagging and boosting approaches. First, we will walk through the required basic concepts. Later we will learn in-depth about these methods. <\/p>\n

Before we drive further, below is the list of concepts you are going to learn in this article.<\/p>\n

What is Ensemble Learning?<\/h2>\n

In machine learning instead of building only a single model<\/a> to predict target or future, how about considering multiple models to predict the target. This is the main idea behind ensemble learning. <\/p>\n

In ensemble learning we will build multiple machine learning models<\/strong><\/a> using the train data, we will discuss how we are going to use the same train data to build various models in the next sections of this article.<\/p>\n

So what advantage will we get with ensemble learning?<\/strong> <\/p><\/blockquote>\n

This is the primary question that will arrive in our mind. <\/p>\n

Let\u2019s pass a second here to think about what advantage<\/strong> we will get if we build multiple models. <\/p>\n

With a single model approach, if the build model is having high bias or high variance<\/strong> we will be limited to that. Even though we are having methods to handle high bias or high variance<\/a>. Still if the final is facing any of the bias or variance issues we can\u2019t do anything.<\/p>\n

Whereas if we  build multiple models we can reduce<\/strong> the high variance and high bias issue by averaging all models. If the individual models are having high bias, then when we build multiple models the high bias will average out. The same is true for high variance cases too.<\/p>\n

For building multiple models we are going to use the same train data.<\/p>\n

If we use the same train data, then all the build models will be also the same right?<\/strong><\/p><\/blockquote>\n

But this is not<\/strong> the case.<\/p>\n

We will learn how to build different models using the same train dataset. Each model will be unique to itself. We will split the available train data into multiple smaller datasets. But while creating these datasets we should follow some key properties<\/strong>. We will talk more about this in the bootstrapping<\/strong> section in this article itself.<\/p>\n

For now just remember, to build multiple models we will split<\/strong> the available train data in smaller datasets. In the next steps, we will learn how to build models using the smaller datasets. One model for one smaller dataset.<\/p>\n

In short: <\/h4>\n

The ensemble learning means instead of building a single model for prediction. We will build multiple machine learning models, we call these models as weak learners<\/strong>. A combination of all weak learners makes the strong learner, Which  generalizes to predict all the target classes with a decent amount of accuracy.<\/p>\n

Different Ensemble Methods <\/h3>\n

\"Ensemble

Ensemble Learning Methods<\/p>\n

We are saying we will build multiple models, how these models will differ from one other. We have two possibilities.<\/p>\n