{"id":1758,"date":"2017-02-06T17:55:43","date_gmt":"2017-02-06T17:55:43","guid":{"rendered":"http:\/\/dataaspirant.com\/?p=1758"},"modified":"2023-12-06T13:59:19","modified_gmt":"2023-12-06T08:29:19","slug":"naive-bayes-classifier-machine-learning","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/naive-bayes-classifier-machine-learning\/","title":{"rendered":"How the Naive Bayes Classifier works in Machine Learning"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><div id=\"attachment_1968\" style=\"width: 610px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1968\" class=\"wp-image-1968 size-full lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier.jpg\" alt=\"Naive Bayes Classifier Algorithm\" width=\"600\" height=\"600\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier.jpg 600w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier-150x150.jpg 150w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier-300x300.jpg 300w\" data-sizes=\"(max-width: 600px) 100vw, 600px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 600px; --smush-placeholder-aspect-ratio: 600\/600;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1968\" class=\"wp-image-1968 size-full\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier.jpg\" alt=\"Naive Bayes Classifier Algorithm\" width=\"600\" height=\"600\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier.jpg 600w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier-150x150.jpg 150w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Naive-Bayes-Classifier-300x300.jpg 300w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/noscript><p id=\"caption-attachment-1968\" class=\"wp-caption-text\">Naive Bayes Classifier Algorithm<\/p><\/div>\n<p style=\"text-align: justify;\">Naive Bayes classifier is a straightforward and powerful algorithm for the\u00a0<a href=\"https:\/\/dataaspirant.com\/2016\/09\/24\/classification-clustering-alogrithms\/\" target=\"_blank\" rel=\"noopener\"><strong>classification<\/strong><\/a> task. Even if we are working on a data set with millions of records with some attributes, it is suggested to try Naive Bayes approach.<\/p>\n<p style=\"text-align: justify;\">Naive Bayes classifier gives great results when we use it for textual data analysis. Such as Natural Language Processing.<\/p>\n<p style=\"text-align: justify;\">To understand the naive Bayes classifier we need to understand the Bayes theorem. So let&#8217;s first discuss the Bayes Theorem.<\/p>\n<p style=\"text-align: justify;\"><!--more--><\/p>\n<p style=\"text-align: justify;\"><span class='bctt-click-to-tweet'><span class='bctt-ctt-text'><a href='https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fdataaspirant.com%2F%3Fp%3D1758&#038;text=How%20Naive%20Bayes%20classifier%20algorithm%20works%20in%20machine%20learning%20&#038;via=dataaspirant&#038;related=dataaspirant' target='_blank'rel=\"noopener noreferrer\">How Naive Bayes classifier algorithm works in machine learning  <\/a><\/span><a href='https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fdataaspirant.com%2F%3Fp%3D1758&#038;text=How%20Naive%20Bayes%20classifier%20algorithm%20works%20in%20machine%20learning%20&#038;via=dataaspirant&#038;related=dataaspirant' target='_blank' class='bctt-ctt-btn'rel=\"noopener noreferrer\">Click To Tweet<\/a><\/span><\/p>\n<h3>What is Bayes Theorem?<\/h3>\n<p style=\"text-align: justify;\">Bayes theorem named after Rev. Thomas Bayes. It works on conditional<strong> probability<\/strong>. Conditional probability is the probability that something will happen, <strong><em>given that something else<\/em> has already occurred<\/strong>. Using the conditional probability, we can calculate the probability of an event using its prior knowledge.<\/p>\n<p style=\"text-align: justify;\">Below is the formula for calculating the conditional probability.<\/p>\n<p>$latex \\textrm{P(H \\textbar E) = } \u00a0\\frac{\\textrm{ P(E \\textbar H) * P(H)}} {\\textrm{P(E)}}$<\/p>\n<p style=\"text-align: justify;\"><strong>where <\/strong><\/p>\n<ul>\n<li style=\"text-align: justify;\">P(H) is the probability of hypothesis H being true. This is known as the\u00a0prior probability.<\/li>\n<li style=\"text-align: justify;\">P(E) is the probability of the evidence(regardless of the\u00a0hypothesis).<\/li>\n<li style=\"text-align: justify;\">P(E|H) is the probability of the evidence given that hypothesis is true.<\/li>\n<li style=\"text-align: justify;\">P(H|E) is the probability of the\u00a0hypothesis given that the\u00a0evidence is there.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Let&#8217;s consider an example to understand how the above formula of Bayes theorem works.<\/p>\n<h4 style=\"text-align: justify;\">Problem:<\/h4>\n<p style=\"text-align: justify;\">A Path Lab is performing a Test of disease say &#8220;D&#8221; with two results &#8220;Positive&#8221; &amp; &#8220;Negative.&#8221; They guarantee that their test result is 99% accurate: if you have the disease, they will give test positive 99% of the time. If you don&#8217;t have the disease, they will test negative 99% of the time. If 3% of all the people have this disease and test gives &#8220;positive&#8221; result, <strong>what is the probability that you actually have the disease?<\/strong><\/p>\n<p style=\"text-align: justify;\">For solving the above problem, we will have to use conditional probability.<br \/>\nProbability of people suffering from Disease D, P(D) = 0.03 = 3%<br \/>\nProbability that test gives &#8220;positive&#8221; result and patient have the disease, P(Pos | D) = 0.99 =99%<\/p>\n<p style=\"text-align: justify;\">Probability of people not suffering from Disease D, P(~D) = 0.97 = 97%<br \/>\nProbability that test gives &#8220;positive&#8221; result and patient does have the disease, P(Pos | ~D) = 0.01 =1%<\/p>\n<p style=\"text-align: justify;\">For calculating the probability that the patient actually have the disease i.e, <strong>P( D | Pos)<\/strong> we will use Bayes theorem:<\/p>\n<p>$latex \\textrm{P(D \\textbar Pos) = } \u00a0\\frac{\\textrm{ P(Pos \\textbar D) * P(D)}} {\\textrm{P(Pos)}}$<\/p>\n<p>&nbsp;<\/p>\n<p>We have all the values of numerator but we need to calculate P(Pos):<br \/>\n<strong>P(Pos)<\/strong> = P(D, pos) + P( ~D, pos)<br \/>\n= P(pos|D)*P(D) + P(pos|~D)*P(~D)<br \/>\n= 0.99 * 0.03 + 0.01 * 0.97<br \/>\n= 0.0297 +\u00a00.0097<br \/>\n= 0.0394<\/p>\n<p>Let&#8217;s calculate, <strong>P( D | Pos) = (P(Pos | D) * P(D)) \/ P(Pos)<\/strong><br \/>\n= (0.99 * 0.03) \/ 0.0394<br \/>\n=\u00a00.753807107<\/p>\n<p>So, Approximately <strong><span style=\"text-decoration: underline;\">75%<\/span><\/strong> chances are there that the patient is actually suffering from disease.<\/p>\n<p>I hope we understand the\u00a0Bayes theorem. Now let&#8217;s use this understanding to find out more about the naive Bayes classifier.<\/p>\n<h2>Naive Bayes Classifier<\/h2>\n<p style=\"text-align: justify;\">Naive Bayes is a kind of classifier which uses the Bayes Theorem. It predicts membership probabilities for each class such as the probability that given record or data point belongs to a particular class. \u00a0The class with the\u00a0highest probability is considered as the\u00a0most likely class. This is also known as <strong>Maximum A Posteriori (MAP)<\/strong>.<\/p>\n<h4 style=\"text-align: justify;\">The MAP for a hypothesis is:<\/h4>\n<p style=\"text-align: left;\"><strong>MAP(H)<\/strong><br \/>\n= max( P(H|E) )<br \/>\n= \u00a0max( (P(E|H)*P(H))\/P(E))<br \/>\n= max(P(E|H)*P(H))<\/p>\n<p style=\"text-align: justify;\">P(E) is evidence probability, and it is used to normalize the result. It remains same so, removing it won&#8217;t affect.<\/p>\n<p style=\"text-align: justify;\">Naive Bayes classifier assumes that\u00a0all the features are<strong>\u00a0unrelated<\/strong> to each other. Presence or absence of a feature does not influence the presence or absence of any other feature. We can use Wikipedia example for explaining the logic i.e.,<\/p>\n<blockquote>\n<p style=\"text-align: justify;\">A fruit may be considered to be an apple if it is red, round, and about 4&#8243; in diameter.\u00a0 Even if these features depend on each other or upon the existence of the other features, a naive Bayes classifier considers all of these properties to independently contribute to the probability that this fruit is an apple.<\/p>\n<\/blockquote>\n<p style=\"text-align: justify;\">In real datasets, we test a hypothesis given multiple evidence(feature). So, calculations become complicated. To simplify the work,\u00a0the feature independence approach is used to &#8216;uncouple&#8217; multiple evidence\u00a0and treat each as an independent one.<\/p>\n<p style=\"text-align: justify;\"><strong>P(H|Multiple Evidences)<\/strong> = \u00a0P(E1| H)* P(E2|H) &#8230;&#8230;*P(En|H) * P(H) \/ P(Multiple Evidences)<\/p>\n<h2 style=\"text-align: left;\">Example of Naive Bayes Classifier<\/h2>\n<p style=\"text-align: justify;\">For understanding a theoretical concept, the best procedure is to try it on an example. Since\u00a0I am a pet lover so selected animals as our predicted class. \ud83d\ude42<\/p>\n<div id=\"attachment_1952\" style=\"width: 810px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1952\" class=\"size-full wp-image-1952 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features.png\" alt=\"Naive bayes classification features\" width=\"800\" height=\"800\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features-150x150.png 150w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features-300x300.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features-768x768.png 768w\" data-sizes=\"(max-width: 800px) 100vw, 800px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 800px; --smush-placeholder-aspect-ratio: 800\/800;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1952\" class=\"size-full wp-image-1952\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features.png\" alt=\"Naive bayes classification features\" width=\"800\" height=\"800\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features-150x150.png 150w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features-300x300.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/features-768x768.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/noscript><p id=\"caption-attachment-1952\" class=\"wp-caption-text\">Naive Bayes classification features<\/p><\/div>\n<p style=\"text-align: justify;\">Let&#8217;s consider a training dataset with 1500 records and 3 classes. We presume that there are no missing values in our data. We have<\/p>\n<h4 style=\"text-align: justify;\">We have <strong>3 classes associated with Animal Types<\/strong>:<\/h4>\n<ul>\n<li style=\"text-align: justify;\">Parrot,<\/li>\n<li style=\"text-align: justify;\">Dog,<\/li>\n<li style=\"text-align: justify;\">Fish.<\/li>\n<\/ul>\n<h4 style=\"text-align: justify;\">The Predictor features set consists of <strong>4 features<\/strong> as<\/h4>\n<ul>\n<li style=\"text-align: justify;\">Swim<\/li>\n<li style=\"text-align: justify;\">Wings<\/li>\n<li style=\"text-align: justify;\">Green Color<\/li>\n<li style=\"text-align: justify;\">Dangerous Teeth.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Green Color, Dangerous Teeth. \u00a0All the features are categorical variables with either of the 2 values: <strong>T(True) or F( False)<\/strong>.<\/p>\n<table style=\"height: 139px; border-color: #000000;\" border=\"1px\" width=\"625\">\n<tbody>\n<tr>\n<td style=\"width: 106px; background-color: #c2bebe;\"><strong>Swim<\/strong><\/td>\n<td style=\"width: 106px; background-color: #c2bebe;\"><strong>Wings<\/strong><\/td>\n<td style=\"width: 107px; background-color: #c2bebe;\"><strong>Green Color<\/strong><\/td>\n<td style=\"width: 106px; background-color: #c2bebe;\"><strong>Dangerous Teeth<\/strong><\/td>\n<td style=\"width: 166px; background-color: #c2bebe;\"><strong>Animal Type<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"font-weight: 400; width: 106px;\">50<\/td>\n<td style=\"font-weight: 400; width: 106px;\">500\/500<\/td>\n<td style=\"font-weight: 400; width: 107px;\">400\/500<\/td>\n<td style=\"font-weight: 400; width: 106px;\">0<\/td>\n<td style=\"font-weight: 400; width: 166px;\">Parrot<\/td>\n<\/tr>\n<tr>\n<td style=\"font-weight: 400; width: 106px;\">450\/500<\/td>\n<td style=\"font-weight: 400; width: 106px;\">0<\/td>\n<td style=\"font-weight: 400; width: 107px;\">0<\/td>\n<td style=\"font-weight: 400; width: 106px;\">500\/500<\/td>\n<td style=\"font-weight: 400; width: 166px;\">Dog<\/td>\n<\/tr>\n<tr>\n<td style=\"font-weight: 400; width: 106px;\">500\/500<\/td>\n<td style=\"font-weight: 400; width: 106px;\">0<\/td>\n<td style=\"font-weight: 400; width: 107px;\">100\/500<\/td>\n<td style=\"font-weight: 400; width: 106px;\">50\/500<\/td>\n<td style=\"font-weight: 400; width: 166px;\">Fish<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p style=\"text-align: justify;\">The above table shows a frequency table of our data. In our training data:<\/p>\n<ul style=\"text-align: justify;\">\n<li style=\"text-align: left;\">Parrots have 50(10%) value for Swim, i.e., 10% parrot can swim according to our data, 500 out of 500(100%) parrots have wings, 400 out of 500(80%) parrots are Green and 0(0%) parrots have Dangerous Teeth.<\/li>\n<li style=\"text-align: left;\">Classes with Animal type Dogs shows that 450 out of 500(90%) can swim, 0(0%) dogs have wings, 0(0%) dogs are of Green color and 500 out of 500(100%) dogs have Dangerous Teeth.<\/li>\n<li style=\"text-align: left;\">Classes with Animal type Fishes shows that 500 out of 500(100%) can swim, 0(0%) fishes have wings, 100(20%) fishes are of Green color and 50 out of 500(10%) dogs have Dangerous Teeth.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Now, it&#8217;s time to work on\u00a0predict classes using the Naive Bayes model. We have taken 2 records that have values in their feature set, but the target variable needs to predicted.<\/p>\n<table style=\"height: 112px; border-color: #000000; width: 624px;\" border=\"1 px\">\n<tbody>\n<tr>\n<td style=\"width: 38px; background-color: #a8a5a5; text-align: center;\"><\/td>\n<td style=\"width: 111px; background-color: #a8a5a5; text-align: center;\"><strong>Swim<\/strong><\/td>\n<td style=\"width: 127px; background-color: #a8a5a5; text-align: center;\"><strong>Wings<\/strong><\/td>\n<td style=\"width: 116px; background-color: #a8a5a5; text-align: center;\"><strong>Green\u00a0<\/strong><\/td>\n<td style=\"width: 150px; background-color: #a8a5a5; text-align: center;\"><strong>Teeth<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"font-weight: 400; width: 38px; text-align: center;\">1.<\/td>\n<td style=\"font-weight: 400; width: 111px; text-align: center;\">True<\/td>\n<td style=\"font-weight: 400; width: 127px; text-align: center;\">False<\/td>\n<td style=\"font-weight: 400; width: 116px; text-align: center;\">True<\/td>\n<td style=\"font-weight: 400; width: 150px; text-align: center;\">False<\/td>\n<\/tr>\n<tr>\n<td style=\"font-weight: 400; width: 38px; text-align: center;\">2.<\/td>\n<td style=\"font-weight: 400; width: 111px; text-align: center;\">True<\/td>\n<td style=\"font-weight: 400; width: 127px; text-align: center;\">False<\/td>\n<td style=\"font-weight: 400; width: 116px; text-align: center;\">True<\/td>\n<td style=\"font-weight: 400; width: 150px; text-align: center;\">True<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p style=\"text-align: left;\">We have to predict animal type using the feature values. We have to predict whether the animal is a Dog, a Parrot or a Fish<\/p>\n<p style=\"text-align: left;\">We will use the Naive Bayes approach<br \/>\n<strong>P(H|Multiple Evidences)<\/strong> = \u00a0P(E1| H)* P(E2|H) &#8230;&#8230;*P(En|H) * P(H) \/ P(Multiple Evidences)<\/p>\n<p style=\"text-align: left;\">Let&#8217;s consider the first record.<br \/>\nThe Evidence here is <strong>Swim &amp; Green<\/strong>. The Hypothesis can be an animal type to be Dog, Parrot, Fish.<\/p>\n<h4 style=\"text-align: left;\">For Hypothesis testing for the animal to be a Dog:<\/h4>\n<p style=\"text-align: left;\">P(Dog | Swim, Green) = P(Swim|Dog) * P(Green|Dog) * P(Dog) \/ P(Swim, Green)<br \/>\n= \u00a00.9 * 0 * 0.333 \/\u00a0P(Swim, Green)<br \/>\n= 0<\/p>\n<h4 style=\"text-align: left;\">For Hypothesis testing for the animal to be a Parrot:<\/h4>\n<p style=\"text-align: left;\">P(Parrot| Swim, Green) = P(Swim|Parrot) * P(Green|Parrot) * P(Parrot) \/ P(Swim, Green)<br \/>\n= \u00a00.1 * 0.80 * 0.333 \/\u00a0P(Swim, Green)<br \/>\n=\u00a00.0264\/ P(Swim, Green)<\/p>\n<h4 style=\"text-align: left;\">For Hypothesis testing for the animal to be a Fish:<\/h4>\n<p style=\"text-align: left;\">P(Fish| Swim, Green) = P(Swim|Fish) * P(Green|Fish) * P(Fish) \/ P(Swim, Green)<br \/>\n= \u00a01 * 0.2 * 0.333 \/\u00a0P(Swim, Green)<br \/>\n=\u00a00.0666\/\u00a0P(Swim, Green)<\/p>\n<p style=\"text-align: left;\">The denominator of all the above calculations is same i.e, P(Swim, Green). The value of\u00a0P(Fish| Swim, Green) is greater that\u00a0P(Parrot| Swim, Green).<\/p>\n<p style=\"text-align: left;\">Using Naive Bayes, we can predict that the class of this record is <span style=\"text-decoration: underline;\">Fish<\/span>.<\/p>\n<p style=\"text-align: left;\">Let&#8217;s consider the second record.<br \/>\nThe Evidence here is Swim, Green &amp; Teeth. The Hypothesis can be an animal type to be Dog, Parrot, Fish.<\/p>\n<h4 style=\"text-align: left;\">For Hypothesis testing for the animal to be a Dog:<\/h4>\n<p style=\"text-align: left;\">P(Dog | Swim, Green, Teeth) = P(Swim|Dog) * P(Green|Dog) * P(Teeth|Dog) * P(Dog) \/ P(Swim, Green, Teeth)<br \/>\n= \u00a00.9 * 0 * 1 * 0.333 \/\u00a0P(Swim, Green, Teeth)<br \/>\n= 0<\/p>\n<h4 style=\"text-align: left;\">For Hypothesis testing for the animal to be a Parrot:<\/h4>\n<p style=\"text-align: left;\">P(Parrot| Swim, Green, Teeth) = P(Swim|Parrot) * P(Green|Parrot)* P(Teeth|Parrot) * P(Parrot) \/ P(Swim, Green, Teeth)<br \/>\n= \u00a00.1 * 0.80 * \u00a00 *0.333 \/\u00a0P(Swim, Green, Teeth)<br \/>\n= 0<\/p>\n<h4 style=\"text-align: left;\">For Hypothesis testing for the animal to be a Fish:<\/h4>\n<p style=\"text-align: left;\">P(Fish|Swim, Green, Teeth) = P(Swim|Fish) * P(Green|Fish) * P(Teeth|Fish) *P(Fish) \/ P(Swim, Green, Teeth)<br \/>\n= \u00a01 * 0.2 * 0.1 * 0.333 \/\u00a0P(Swim, Green, Teeth)<br \/>\n=\u00a00.00666 \/\u00a0P(Swim, Green, Teeth)<\/p>\n<p style=\"text-align: justify;\">The denominator of all the above calculations is same i.e, P(Swim, Green, Teeth). The value of\u00a0P(Fish| Swim, Green, Teeth) is the only positive value greater than 0. Using Naive Bayes, we can predict that the class of this record is <span style=\"text-decoration: underline;\">Fish<\/span>.<\/p>\n<p style=\"text-align: justify;\">As the calculated value of probabilities is very less. To normalize these values, we need to use denominators.<\/p>\n<p style=\"text-align: justify;\">Let&#8217;s proceed to learn the various type of <strong>Naive Bayes Methods<\/strong>.<\/p>\n<h2 style=\"text-align: left;\">Types of Naive Bayes Algorithm<\/h2>\n<h3 style=\"text-align: left;\">\u00a0Gaussian Naive Bayes<\/h3>\n<p style=\"text-align: justify;\">When attribute values are\u00a0continuous, an assumption is made that the values associated with each class are distributed according\u00a0to Gaussian i.e., Normal Distribution.<\/p>\n<p style=\"text-align: justify;\">If in our data, an attribute say &#8220;x&#8221;\u00a0contains continuous data. We first segment the data by the class and then compute mean $latex \\mu_{y} $\u00a0&amp; Variance $latex {\\sigma_{y}}^{2} $ of each class.<br \/>\n<img decoding=\"async\" data-src=\"http:\/\/scikit-learn.org\/stable\/_images\/math\/ed0c1181c1696f72e1be266187e4694919047d9e.png\" alt=\"P(x_i \\mid y) &amp;= \\frac{1}{\\sqrt{2\\pi\\sigma^2_y}} \\exp\\left(-\\frac{(x_i - \\mu_y)^2}{2\\sigma^2_y}\\right)\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" \/><noscript><img decoding=\"async\" src=\"http:\/\/scikit-learn.org\/stable\/_images\/math\/ed0c1181c1696f72e1be266187e4694919047d9e.png\" alt=\"P(x_i \\mid y) &amp;= \\frac{1}{\\sqrt{2\\pi\\sigma^2_y}} \\exp\\left(-\\frac{(x_i - \\mu_y)^2}{2\\sigma^2_y}\\right)\" \/><\/noscript><\/p>\n<h3>MultiNomial Naive Bayes<\/h3>\n<p style=\"text-align: justify;\">MultiNomial Naive Bayes is preferred to use on data that is multinomially distributed. It is one of the standard\u00a0classic algorithms. Which\u00a0is used in text categorization (classification). Each event in text classification represents the occurrence of a word in a document.<\/p>\n<h3>Bernoulli Naive Bayes<\/h3>\n<p style=\"text-align: justify;\">Bernoulli Naive Bayes is used on the data that is distributed\u00a0according to multivariate Bernoulli distributions.i.e., multiple features can be there, but each one is assumed to be a binary-valued (Bernoulli, boolean) variable. So, it requires features to be binary valued.<\/p>\n<h3>Advantages and Disadvantage of Naive Bayes classifier<\/h3>\n<h4>Advantages<\/h4>\n<ul>\n<li>Naive Bayes Algorithm is a fast, highly scalable algorithm.<\/li>\n<li>Naive Bayes can be use for Binary and Multiclass classification. It provides different types of Naive Bayes Algorithms like GaussianNB, MultinomialNB, BernoulliNB.<\/li>\n<li>It is a simple algorithm that depends on doing a\u00a0bunch of counts.<\/li>\n<li>Great choice for Text Classification problems. It&#8217;s a popular choice for spam email classification.<\/li>\n<li>It can be easily train on small dataset<\/li>\n<\/ul>\n<h4>Disadvantages<\/h4>\n<ul>\n<li>It considers all the features to be unrelated, so it cannot learn the relationship between features. E.g., Let&#8217;s say Remo is going to a part. While cloth selection for the party, Remo is looking at his cupboard. Remo likes to wear a\u00a0white color shirt. In Jeans, he likes to wear a brown Jeans, But Remo doesn&#8217;t like wearing a white shirt with Brown Jeans.\u00a0Naive Bayes can learn individual features importance but can&#8217;t determine the relationship among features.<\/li>\n<\/ul>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>| <span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span> | <span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>| <span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span> | <span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0| <a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions, then feel free to comment below.\u00a0 If you want me to write on one particular topic, then do tell it to me in the comments below.<\/p>\n<h3>Related Courses:<\/h3>\n<p>Do check out <a href=\"https:\/\/dataaspirant.com\/data-science-courses\/\" target=\"_blank\">unlimited data science courses<\/a><\/p>\n<p>&nbsp;<\/p>\n<table style=\"width: 737px; height: 1202px;\">\n<tbody>\n<tr>\n<td style=\"width: 269.84375px;\"><strong>Title &amp; links<\/strong><\/td>\n<td style=\"width: 203.15625px;\"><strong>Details<\/strong><\/td>\n<td style=\"width: 250px;\"><strong>What You Will Learn<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 269.84375px;\"><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.950390&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fmachinelearning%2F\" target=\"_blank\">Machine Learning A-Z: Hands-On Python &amp; R In Data Science<\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.950390&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><a class=\"hasimg\" href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.950390&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fmachinelearning%2F\" target=\"_blank\"><img loading=\"lazy\" decoding=\"async\" data-src=\"https:\/\/i2.wp.com\/udemy-images.udemy.com\/course\/480x270\/950390_270f_3.jpg?w=690&amp;ssl=1\" width=\"209\" height=\"118\" border=\"0\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 209px; --smush-placeholder-aspect-ratio: 209\/118;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" src=\"https:\/\/i2.wp.com\/udemy-images.udemy.com\/course\/480x270\/950390_270f_3.jpg?w=690&amp;ssl=1\" width=\"209\" height=\"118\" border=\"0\" \/><\/noscript><\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.950390&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/td>\n<td style=\"width: 203.15625px;\"><span class=\"rating-and-enrolled__element\">Students Enrolled::\u00a0<strong>19,359<\/strong><\/span><\/p>\n<p>Course Overall Rating::\u00a0<strong>4.6\u00a0<\/strong><\/td>\n<td style=\"width: 250px;\">\n<ul class=\"regular-list\">\n<li>Master Machine Learning on Python &amp; R<\/li>\n<li>Make robust Machine Learning models.<\/li>\n<li>Handle specific topics like Reinforcement Learning, NLP and <a href=\"https:\/\/dataaspirant.com\/deep-learning-topics\/\" target=\"_blank\">Deep Learning<\/a>.<\/li>\n<li>Build an army of powerful Machine Learning models and know how to combine them to solve any problem.<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 269.84375px;\"><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=479491.1560515878&amp;type=2&amp;murl=https%3A%2F%2Fwww.coursera.org%2Flearn%2Fml-classification\" target=\"_blank\">Machine Learning: Classification<\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=479491.1560515878&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=479491.1560515878&amp;type=2&amp;murl=https%3A%2F%2Fwww.coursera.org%2Flearn%2Fml-classification\" target=\"_blank\"><img decoding=\"async\" data-src=\"https:\/\/d3njjcbhbojbot.cloudfront.net\/api\/utilities\/v1\/imageproxy\/https:\/\/coursera-course-photos.s3.amazonaws.com\/b1\/37a1a04d0011e59bd777c2a1ef6a6e\/gears-818461_1280.jpg\" border=\"0\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" \/><noscript><img decoding=\"async\" src=\"https:\/\/d3njjcbhbojbot.cloudfront.net\/api\/utilities\/v1\/imageproxy\/https:\/\/coursera-course-photos.s3.amazonaws.com\/b1\/37a1a04d0011e59bd777c2a1ef6a6e\/gears-818461_1280.jpg\" border=\"0\" \/><\/noscript><\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=479491.1560515878&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/td>\n<td style=\"width: 203.15625px;\">Course Overall Rating::\u00a0<strong>4.7<\/strong><\/td>\n<td style=\"width: 250px;\">\n<ul>\n<li>Describe the input and output of a classification model.<\/li>\n<li>Build a classification model to predict sentiment in a product review dataset.<\/li>\n<li>Analyze financial data to predict loan defaults.<\/li>\n<\/ul>\n<ul>\n<li>Evaluate your models using precision-recall metrics.<\/li>\n<li>Implement these techniques in Python (or in the language of your choice, though Python is highly recommended).<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 269.84375px;\"><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=479491.1560519184&amp;type=2&amp;murl=https%3A%2F%2Fwww.coursera.org%2Flearn%2Fpractical-machine-learning\" target=\"_blank\">Practical Machine Learning<\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=479491.1560519184&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><br \/>\n<a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=479491.1560519184&amp;type=2&amp;murl=https%3A%2F%2Fwww.coursera.org%2Flearn%2Fpractical-machine-learning\" target=\"_blank\"><img decoding=\"async\" data-src=\"https:\/\/d3njjcbhbojbot.cloudfront.net\/api\/utilities\/v1\/imageproxy\/https:\/\/coursera-course-photos.s3.amazonaws.com\/a1\/e7472069b611e3ae92c39913bb30e0\/PredictionMachineLearning.jpg\" border=\"0\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" \/><noscript><img decoding=\"async\" src=\"https:\/\/d3njjcbhbojbot.cloudfront.net\/api\/utilities\/v1\/imageproxy\/https:\/\/coursera-course-photos.s3.amazonaws.com\/a1\/e7472069b611e3ae92c39913bb30e0\/PredictionMachineLearning.jpg\" border=\"0\" \/><\/noscript><\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=479491.1560519184&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/td>\n<td style=\"width: 203.15625px;\">Course Overall\u00a0Rating::\u00a0<strong>4.4<\/strong><\/td>\n<td style=\"width: 250px;\">\n<ul>\n<li>This course will cover the basic components of building and applying prediction functions with an emphasis on practical applications.<\/li>\n<li>Will provide the\u00a0basic grounding in concepts such as training and tests sets, overfitting, and error rates and introduce a range of model-based and algorithmic machine learning methods<\/li>\n<li>These methods including regression, classification trees, Naive Bayes, and random forests.<\/li>\n<li>The course will cover the complete process of building prediction functions including data collection, feature creation, algorithms, and evaluation.<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Naive Bayes classifier is a straightforward and powerful algorithm for the\u00a0classification task. Even if we are working on a data set with millions of records with some attributes, it is suggested to try Naive Bayes approach. Naive Bayes classifier gives great results when we use it for textual data analysis. Such as Natural Language Processing.<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/naive-bayes-classifier-machine-learning\/\">+ Read More<\/a><\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[3,6],"tags":[39,718],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1758"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=1758"}],"version-history":[{"count":1,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1758\/revisions"}],"predecessor-version":[{"id":11888,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1758\/revisions\/11888"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=1758"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=1758"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=1758"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}