{"id":1825,"date":"2017-02-20T12:30:06","date_gmt":"2017-02-20T12:30:06","guid":{"rendered":"http:\/\/dataaspirant.com\/?p=1825"},"modified":"2023-12-06T13:53:17","modified_gmt":"2023-12-06T08:23:17","slug":"gaussian-naive-bayes-classifier-implementation-python","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/gaussian-naive-bayes-classifier-implementation-python\/","title":{"rendered":"Gaussian Naive Bayes Classifier implementation in Python"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><div id=\"attachment_2107\" style=\"width: 730px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2107\" class=\"wp-image-2107 size-full lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Gaussian-Naive-Bayes-Python.jpg\" alt=\"Gaussian Naive Bayes classifier Implementation in Python\" width=\"720\" height=\"400\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Gaussian-Naive-Bayes-Python.jpg 720w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Gaussian-Naive-Bayes-Python-300x167.jpg 300w\" data-sizes=\"(max-width: 720px) 100vw, 720px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 720px; --smush-placeholder-aspect-ratio: 720\/400;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2107\" class=\"wp-image-2107 size-full\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Gaussian-Naive-Bayes-Python.jpg\" alt=\"Gaussian Naive Bayes classifier Implementation in Python\" width=\"720\" height=\"400\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Gaussian-Naive-Bayes-Python.jpg 720w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Gaussian-Naive-Bayes-Python-300x167.jpg 300w\" sizes=\"(max-width: 720px) 100vw, 720px\" \/><\/noscript><p id=\"caption-attachment-2107\" class=\"wp-caption-text\">Gaussian Naive Bayes classifier Implementation in Python<\/p><\/div>\n<h2 style=\"text-align: justify;\">Building Gaussian Naive Bayes Classifier in Python<\/h2>\n<p style=\"text-align: justify;\">In this post,\u00a0we are going to implement the <strong>Naive Bayes classifier<\/strong> in Python using my favorite machine learning library scikit-learn. Next, we are going to use the trained Naive Bayes (<a href=\"https:\/\/dataaspirant.com\/2014\/09\/19\/supervised-and-unsupervised-learning\/\" target=\"_blank\">supervised classification<\/a>), model to predict the Census Income.<\/p>\n<p style=\"text-align: justify;\">As we discussed the Bayes theorem in <a href=\"https:\/\/dataaspirant.com\/2017\/02\/06\/naive-bayes-classifier-machine-learning\/\" target=\"_blank\">naive Bayes classifier<\/a> post. We hope you know the basics of the Bayes theorem. If not let&#8217;s quickly look at the basics of Bayes theorem once.<\/p>\n<p style=\"text-align: justify;\"><!--more--><\/p>\n<p style=\"text-align: justify;\">Bayes&#8217; theorem\u00a0is based on <strong>conditional probability<\/strong>. The conditional probability\u00a0helps us calculating the probability that something will happen, <em>given that something else<\/em> has already happened. Not getting let&#8217;s understand with few examples.<\/p>\n<h3 style=\"text-align: justify;\">Conditional Probability Examples<\/h3>\n<p>Below are the few examples helps to clearly understand the definition of conditional probability.<\/p>\n<ul>\n<li>Purchasing mac book when you already purchased the iPhone.<\/li>\n<li>Having a refreshing drink when you are in the movie theater.<\/li>\n<li>Buying peanuts when you brought a chilled soft drink.<\/li>\n<\/ul>\n<div id=\"attachment_2257\" style=\"width: 510px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2257\" class=\"size-full wp-image-2257 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability.jpg\" alt=\"Conditional Probability Example\" width=\"500\" height=\"500\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability.jpg 500w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability-150x150.jpg 150w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability-300x300.jpg 300w\" data-sizes=\"(max-width: 500px) 100vw, 500px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 500px; --smush-placeholder-aspect-ratio: 500\/500;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2257\" class=\"size-full wp-image-2257\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability.jpg\" alt=\"Conditional Probability Example\" width=\"500\" height=\"500\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability.jpg 500w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability-150x150.jpg 150w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/02\/Conditional_probability-300x300.jpg 300w\" sizes=\"(max-width: 500px) 100vw, 500px\" \/><\/noscript><p id=\"caption-attachment-2257\" class=\"wp-caption-text\">Conditional Probability Example<\/p><\/div>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify;\">Using the Bayes theorem the naive Bayes classifier works. The naive Bayes classifier assumes all the features are independent to each other. Even if the features depend on each other or upon the existence of the other features.\u00a0Naive Bayes classifier considers all of these properties to independently contribute to the probability that the user\u00a0buys the \u00a0MacBook.<\/p>\n<p style=\"text-align: justify;\">To learn the key concepts related to Naive Bayes. You can read our article on\u00a0<a href=\"https:\/\/dataaspirant.com\/2017\/02\/06\/naive-bayes-classifier-machine-learning\/\" target=\"_blank\">Introduction to Naive Bayes<\/a>. This will help you understand the core concepts related to Naive Bayes.<\/p>\n<p style=\"text-align: justify;\">In the\u00a0introduction to Naive Bayes post, we discussed <strong>three popular<\/strong> Naive Bayes algorithms:<\/p>\n<ul>\n<li style=\"text-align: justify;\">Gaussian Naive Bayes,<\/li>\n<li style=\"text-align: justify;\">Multinomial Naive Bayes.<\/li>\n<li style=\"text-align: justify;\">Bernoulli Naive Bayes.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">As a continues to the Naive Bayes algorithm article. Now we are going to implement Gaussian Naive Bayes on a &#8220;Census Income&#8221; <a href=\"https:\/\/archive.ics.uci.edu\/ml\/datasets\/Adult\" target=\"_blank\">dataset.<\/a><\/p>\n<h2>Gaussian Naive Bayes<\/h2>\n<p style=\"text-align: justify;\">A Gaussian Naive Bayes algorithm is a special type of NB algorithm. It&#8217;s specifically used when the features have continuous values. It&#8217;s also assumed that all the features are following a gaussian distribution i.e, normal distribution.<\/p>\n<h2>Census Income Dataset<\/h2>\n<p style=\"text-align: justify;\">Census Income dataset is to predict whether the income of a person &gt;$50K\/yr (greater than $50K\/yr) or\u00a0&lt;=$50K\/yr. The data was collected by Barry Becker from 1994 Census dataset.<\/p>\n<p style=\"text-align: justify;\">This dataset was contributed to UCI repository, and It&#8217;s openly available at this <a href=\"https:\/\/archive.ics.uci.edu\/ml\/datasets\/Adult\" target=\"_blank\">link<\/a>. The dataset consists of 15 columns of a mix of discrete as well as continuous data.<\/p>\n<table style=\"height: 296px; width: 602px; border-color: #000000; background-color: #ffffff;\" border=\"1 px\">\n<tbody>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\"><\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\"><strong>Variable Name<\/strong><\/td>\n<td style=\"width: 425px; height: 24px; text-align: center;\"><strong>Variable Range<\/strong><\/td>\n<\/tr>\n<tr style=\"height: 29px;\">\n<td style=\"width: 37px; height: 29px; text-align: center;\">1.<\/td>\n<td style=\"width: 146px; height: 29px; text-align: center;\">age<\/td>\n<td style=\"width: 425px; height: 29px;\">\u00a0[17 &#8211; 90]<\/td>\n<\/tr>\n<tr style=\"height: 87px;\">\n<td style=\"width: 37px; height: 87px; text-align: center;\">2.<\/td>\n<td style=\"width: 146px; height: 87px; text-align: center;\">workclass<\/td>\n<td style=\"width: 425px; height: 87px;\">[&#8216;State-gov&#8217;, &#8216;Self-emp-not-inc&#8217;, &#8216;Private&#8217;, &#8216;Federal-gov&#8217;, &#8216;Local-gov&#8217;, &#8216;Self-emp-inc&#8217;, &#8216;Without-pay&#8217;, &#8216;Never-worked&#8217;]<\/td>\n<\/tr>\n<tr style=\"height: 30px;\">\n<td style=\"width: 37px; height: 30px; text-align: center;\">3.<\/td>\n<td style=\"width: 146px; height: 30px; text-align: center;\">fnlwgt<\/td>\n<td style=\"width: 425px; height: 30px;\">[77516- 257302]<\/td>\n<\/tr>\n<tr style=\"height: 12px;\">\n<td style=\"width: 37px; height: 12px; text-align: center;\">4.<\/td>\n<td style=\"width: 146px; height: 12px; text-align: center;\">education<\/td>\n<td style=\"width: 425px; height: 12px;\">\n<p class=\"\">[&#8216;Bachelors&#8217;, &#8216;HS-grad&#8217;, &#8217;11th&#8217;, &#8216;Masters&#8217;, &#8216;9th&#8217;, &#8216;Some-college&#8217;, &#8216;Assoc-acdm&#8217;, &#8216;Assoc-voc&#8217;, &#8216;7th-8th&#8217;, &#8216;Doctorate&#8217;, &#8216;Prof-school&#8217;, &#8216;5th-6th&#8217;, &#8217;10th&#8217;, &#8216;1st-4th&#8217;, &#8216;Preschool&#8217;, &#8217;12th&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 31px;\">\n<td style=\"width: 37px; height: 31px; text-align: center;\">5.<\/td>\n<td style=\"width: 146px; height: 31px; text-align: center;\">education_num<\/td>\n<td style=\"width: 425px; height: 31px;\">[1 &#8211; 16]<\/td>\n<\/tr>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\">6.<\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\">marital_status<\/td>\n<td style=\"width: 425px; height: 24px;\">\n<p class=\"\">[&#8216;Never-married&#8217;, &#8216;Married-civ-spouse&#8217;, &#8216;Divorced&#8217;, &#8216;Married-spouse-absent&#8217;, &#8216;Separated&#8217;, &#8216;Married-AF-spouse&#8217;, &#8216;Widowed&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\">7.<\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\">occupation<\/td>\n<td style=\"width: 425px; height: 24px;\">\n<p class=\"\">[&#8216;Adm-clerical&#8217;, &#8216;Exec-managerial&#8217;, &#8216;Handlers-cleaners&#8217;, &#8216;Prof-specialty&#8217;, &#8216;Other-service&#8217;, &#8216;Sales&#8217;, &#8216;Craft-repair&#8217;, &#8216;Transport-moving&#8217;, &#8216;Farming-fishing&#8217;, &#8216;Machine-op-inspct&#8217;, &#8216;Tech-support&#8217;, &#8216;Protective-serv&#8217;, &#8216;Armed-Forces&#8217;, &#8216;Priv-house-serv&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\">8.<\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\">relationship<\/td>\n<td style=\"width: 425px; height: 24px;\">\n<p class=\"\">[&#8216;Not-in-family&#8217;, &#8216;Husband&#8217;, &#8216;Wife&#8217;, &#8216;Own-child&#8217;, &#8216;Unmarried&#8217;, &#8216;Other-relative&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\">9.<\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\">race<\/td>\n<td style=\"width: 425px; height: 24px;\">\n<p class=\"\">[&#8216;White&#8217;, &#8216;Black&#8217;, &#8216;Asian-Pac-Islander&#8217;, &#8216;Amer-Indian-Eskimo&#8217;, &#8216;Other&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\">10.<\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\">sex<\/td>\n<td style=\"width: 425px; height: 24px;\">\n<p class=\"\">[&#8216;Male&#8217;, &#8216;Female&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 29px;\">\n<td style=\"width: 37px; height: 29px; text-align: center;\">11.<\/td>\n<td style=\"width: 146px; height: 29px; text-align: center;\">capital_gain<\/td>\n<td style=\"width: 425px; height: 29px;\">[0 &#8211; 99999]<\/td>\n<\/tr>\n<tr style=\"height: 28px;\">\n<td style=\"width: 37px; height: 28px; text-align: center;\">12.<\/td>\n<td style=\"width: 146px; height: 28px; text-align: center;\">capital_loss<\/td>\n<td style=\"width: 425px; height: 28px;\">[0 &#8211;\u00a04356]<\/td>\n<\/tr>\n<tr style=\"height: 28px;\">\n<td style=\"width: 37px; height: 28px; text-align: center;\">13.<\/td>\n<td style=\"width: 146px; height: 28px; text-align: center;\">hours_per_week<\/td>\n<td style=\"width: 425px; height: 28px;\">[1 &#8211; 99]<\/td>\n<\/tr>\n<tr style=\"height: 24px;\">\n<td style=\"width: 37px; height: 24px; text-align: center;\">14.<\/td>\n<td style=\"width: 146px; height: 24px; text-align: center;\">native_country<\/td>\n<td style=\"width: 425px; height: 24px;\">\n<p class=\"\">[&#8216;United-States&#8217;, &#8216;Cuba&#8217;, &#8216;Jamaica&#8217;, &#8216;India&#8217;, &#8216;Mexico&#8217;, &#8216;South&#8217;, &#8216;Puerto-Rico&#8217;, &#8216;Honduras&#8217;, &#8216;England&#8217;, &#8216;Canada&#8217;, &#8216;Germany&#8217;, &#8216;Iran&#8217;, &#8216;Philippines&#8217;, &#8216;Italy&#8217;, &#8216;Poland&#8217;, &#8216;Columbia&#8217;, &#8216;Cambodia&#8217;, &#8216;Thailand&#8217;, &#8216;Ecuador&#8217;, &#8216;Laos&#8217;, &#8216;Taiwan&#8217;, &#8216;Haiti&#8217;, &#8216;Portugal&#8217;, &#8216;Dominican-Republic&#8217;, &#8216;El-Salvador&#8217;, &#8216;France&#8217;, &#8216;Guatemala&#8217;, &#8216;China&#8217;, &#8216;Japan&#8217;, &#8216;Yugoslavia&#8217;, &#8216;Peru&#8217;, &#8216;Outlying-US(Guam-USVI-etc)&#8217;, &#8216;Scotland&#8217;, &#8216;Trinadad&amp;Tobago&#8217;, &#8216;Greece&#8217;, &#8216;Nicaragua&#8217;, &#8216;Vietnam&#8217;, &#8216;Hong&#8217;, &#8216;Ireland&#8217;, &#8216;Hungary&#8217;, &#8216;Holand-Netherlands&#8217;]<\/p>\n<\/td>\n<\/tr>\n<tr style=\"height: 13.5938px;\">\n<td style=\"width: 37px; height: 13.5938px; text-align: center;\">15.<\/td>\n<td style=\"width: 146px; height: 13.5938px; text-align: center;\">income<\/td>\n<td style=\"width: 425px; height: 13.5938px;\">\n<p class=\"\">\u00a0[&#8216;&lt;=50K&#8217;, &#8216;&gt;50K&#8217;]<\/p>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The final <strong>target variable<\/strong> consists of two values: &#8216;&lt;=50K&#8221; &amp; &#8216;&gt;50K&#8217;.<\/p>\n<h2>Implementation of Gaussian NB\u00a0on Census Income dataset<\/h2>\n<h3>Importing Python Machine Learning Libraries<\/h3>\n<p style=\"text-align: justify;\">We need to import <a href=\"https:\/\/en.wikipedia.org\/wiki\/Pandas_(software)\" target=\"_blank\">pandas<\/a>, <a href=\"https:\/\/en.wikipedia.org\/wiki\/NumPy\" target=\"_blank\">numpy<\/a> and <a href=\"http:\/\/scikit-learn.org\/\" target=\"_blank\">sklearn libraries<\/a>. From sklearn, we need to import preprocessing modules like Imputer. The Imputer package\u00a0helps to impute the missing values.<\/p>\n<p style=\"text-align: justify;\">If you are not setup the <a href=\"https:\/\/dataaspirant.com\/2014\/11\/01\/python-packages-for-datamining\/\" target=\"_blank\">python machine learning libraries setup<\/a>. You can first complete it to run the codes in this articles.<\/p>\n<pre class=\"toolbar-overlay:false toolbar-hide:false toolbar-delay:false lang:python decode:true \" title=\"Import required packages\"># Required Python Machine learning Packages\r\nimport pandas as pd\r\nimport numpy as np\r\n# For preprocessing the data\r\nfrom sklearn.preprocessing import Imputer\r\nfrom sklearn import preprocessing\r\n# To split the dataset into train and test datasets\r\nfrom sklearn.cross_validation import train_test_split\r\n# To model the Gaussian Navie Bayes classifier\r\nfrom sklearn.naive_bayes import GaussianNB\r\n# To calculate the accuracy score of the model\r\nfrom sklearn.metrics import accuracy_score<\/pre>\n<p>The <strong>train_test_split<\/strong> module is for splitting the dataset into training and testing set. The <strong>accuracy_score module<\/strong> will be used for calculating the accuracy of our Gaussian Naive Bayes algorithm.<\/p>\n<h3>Data Import<\/h3>\n<p style=\"text-align: justify;\">For importing the census data, we are using pandas <strong>read_csv()<\/strong> method. This method is a\u00a0very simple and fast method for importing data.<\/p>\n<p style=\"text-align: justify;\">We are passing four parameters. The &#8216;adult.data&#8217; parameter is the <strong>file name<\/strong>. The header parameter is for giving details to pandas that whether\u00a0the first row of data consists of <strong>headers or not<\/strong>.\u00a0In our dataset, there is no header. So, we are passing <strong>None<\/strong>.<\/p>\n<p style=\"text-align: justify;\">The delimiter parameter is for giving the information the delimiter that is separating the data. Here, we are using<strong>\u00a0&#8216; *, *&#8217;<\/strong> delimiter. This delimiter is to show delete the spaces before and after the data values. This is very helpful when\u00a0there is inconsistency in spaces used with data values.<\/p>\n<pre class=\"lang:python decode:true\" title=\"Loading the Dataset\">adult_df = pd.read_csv('adult.data',\r\n                       header = None, delimiter=' *, *', engine='python')<\/pre>\n<p style=\"text-align: justify;\">Let&#8217;s <strong>add headers<\/strong> to our dataframe. The below code snippet can be used to perform this task.<\/p>\n<div class=\"CodeMirror-lines\">\n<div class=\"CodeMirror-measure\">\n<pre class=\"lang:python decode:true\" title=\"Adding the headers\">adult_df.columns = ['age', 'workclass', 'fnlwgt', 'education', 'education_num',\r\n                    'marital_status', 'occupation', 'relationship',\r\n                    'race', 'sex', 'capital_gain', 'capital_loss',\r\n                    'hours_per_week', 'native_country', 'income']<\/pre>\n<h3>Handling Missing Data<\/h3>\n<p>Let&#8217;s try to test whether there is any <strong>null value<\/strong> in our dataset or not. We can do this using <strong>isnull()<\/strong> method.<\/p>\n<\/div>\n<div class=\"CodeMirror-measure\">\n<pre class=\"lang:python decode:true\" title=\"Finding Null values\">adult_df.isnull().sum()<\/pre>\n<h4>Script Output<\/h4>\n<pre class=\"lang:default decode:true\">age               0\r\nworkclass         0\r\nfnlwgt            0\r\neducation         0\r\neducation_num     0\r\nmarital_status    0\r\noccupation        0\r\nrelationship      0\r\nrace              0\r\nsex               0\r\ncapital_gain      0\r\ncapital_loss      0\r\nhours_per_week    0\r\nnative_country    0\r\nincome            0\r\ndtype: int64<\/pre>\n<\/div>\n<\/div>\n<p style=\"text-align: justify;\">The above output shows that there is <strong>no &#8220;null&#8221;<\/strong> value in our dataset.<\/p>\n<p style=\"text-align: justify;\">Let&#8217;s try to test whether any categorical attribute contains a &#8220;?&#8221; in it or not.\u00a0At times there exists &#8220;?&#8221; or &#8221; &#8221; in place of missing values. Using the below code snippet we are going to test whether our adult_df data frame consists of categorical variables with values as &#8220;?&#8221;.<\/p>\n<pre class=\"lang:python decode:true\">for value in ['workclass', 'education',\r\n          'marital_status', 'occupation',\r\n          'relationship','race', 'sex',\r\n          'native_country', 'income']:\r\n    print value,\":\", sum(adult_df[value] == '?')<\/pre>\n<h4>Script Output<\/h4>\n<pre class=\"lang:default decode:true\">workclass : 1836\r\neducation : 0\r\nmarital_status : 0\r\noccupation : 1843\r\nrelationship : 0\r\nrace : 0\r\nsex : 0\r\nnative_country : 583\r\nincome : 0<\/pre>\n<p style=\"text-align: justify;\">The output of the above code snippet shows that there are <strong>1836<\/strong> missing values in <strong>workclass<\/strong> attribute. 1843 missing values in occupation attribute and <strong>583<\/strong> values in <strong>native_country<\/strong> attribute.<\/p>\n<h3 style=\"text-align: justify;\">Data preprocessing<\/h3>\n<p style=\"text-align: justify;\">For preprocessing, we are going to make a duplicate copy of our original dataframe.We are duplicating adult_df to adult_df_rev dataframe.<\/p>\n<pre class=\"lang:default decode:true\">adult_df_rev = adult_df<\/pre>\n<p style=\"text-align: justify;\">As we want to\u00a0perform some imputation for missing values. Before doing that, we need some summary <a href=\"https:\/\/dataaspirant.com\/statistics-guide\/\" target=\"_blank\">statistics<\/a> of our dataframe. For this, we can use <strong>describe()<\/strong> method. It can be used to generate various summary statistics, excluding<strong> NaN<\/strong> values.<\/p>\n<p style=\"text-align: justify;\">We are passing an &#8220;include&#8221; parameter with value as &#8220;all&#8221;, this is used to specify that. we want <strong>summary statistics<\/strong> of all the attributes.<\/p>\n<pre class=\"lang:python decode:true\" title=\"Dataset basic statistics\">adult_df_rev.describe(include= 'all')\r\n<\/pre>\n<div class=\"CodeMirror-lines\">You check the <strong>basic statistics<\/strong> about the dataset after running the above command. You can spend some time here to get in details about each and ever stats provided.<\/div>\n<div class=\"CodeMirror-lines\"><\/div>\n<h3 class=\"CodeMirror-lines\">Data Imputation Step<\/h3>\n<div class=\"CodeMirror-lines\" style=\"text-align: justify;\">Now, it&#8217;s time to impute the missing values. Some of our categorical values have missing values i.e, &#8220;?&#8221;. We are going to replace the &#8220;?&#8221; with the above describe methods top row&#8217;s value. For example, we\u00a0are going to replace the &#8220;?&#8221; values of workplace attribute with &#8220;Private&#8221; value.<\/div>\n<div class=\"CodeMirror-lines\">\n<pre class=\"lang:python decode:true\" title=\"Data Imputation\">for value in ['workclass', 'education',\r\n          'marital_status', 'occupation',\r\n          'relationship','race', 'sex',\r\n          'native_country', 'income']:\r\n    adult_df_rev[value].replace(['?'], [adult_df_rev.describe(include='all')[value][2]],\r\n                                inplace='True')<\/pre>\n<p>We have performed data imputation step. \ud83d\ude42<br \/>\nYou can check changes in dataframe by printing\u00a0<span class=\"lang:default decode:true crayon-inline\">adult_df_rev<\/span><\/p>\n<\/div>\n<div class=\"CodeMirror-lines\" style=\"text-align: justify;\">For naive Bayes, we need to convert all the data values in one format. We are going to encode all the labels with the\u00a0value between<strong> 0<\/strong> and <strong>n_classes-1<\/strong>.<\/div>\n<div class=\"CodeMirror-lines\" style=\"text-align: justify;\"><\/div>\n<h3 class=\"CodeMirror-lines\" style=\"text-align: justify;\">One-Hot Encoder<\/h3>\n<div class=\"CodeMirror-lines\" style=\"text-align: justify;\">For implementing this, we are going to use <a href=\"http:\/\/scikit-learn.org\/stable\/modules\/generated\/sklearn.preprocessing.LabelEncoder.html\" target=\"_blank\">LabelEncoder <\/a>of scikit learn library. For encoding, we can also use the\u00a0One-Hot encoder. It encodes the data into binary format.<\/div>\n<div class=\"CodeMirror-lines\">\n<pre class=\"lang:python decode:true\" title=\"One Hot Encoding\">le = preprocessing.LabelEncoder()\r\nworkclass_cat = le.fit_transform(adult_df.workclass)\r\neducation_cat = le.fit_transform(adult_df.education)\r\nmarital_cat   = le.fit_transform(adult_df.marital_status)\r\noccupation_cat = le.fit_transform(adult_df.occupation)\r\nrelationship_cat = le.fit_transform(adult_df.relationship)\r\nrace_cat = le.fit_transform(adult_df.race)\r\nsex_cat = le.fit_transform(adult_df.sex)\r\nnative_country_cat = le.fit_transform(adult_df.native_country)<\/pre>\n<pre class=\"lang:python decode:true\" title=\"One hot Encoding\">#initialize the encoded categorical columns\r\nadult_df_rev['workclass_cat'] = workclass_cat\r\nadult_df_rev['education_cat'] = education_cat\r\nadult_df_rev['marital_cat'] = marital_cat\r\nadult_df_rev['occupation_cat'] = occupation_cat\r\nadult_df_rev['relationship_cat'] = relationship_cat\r\nadult_df_rev['race_cat'] = race_cat\r\nadult_df_rev['sex_cat'] = sex_cat\r\nadult_df_rev['native_country_cat'] = native_country_cat<\/pre>\n<pre class=\"lang:python decode:true\">#drop the old categorical columns from dataframe\r\ndummy_fields = ['workclass', 'education', 'marital_status', \r\n                  'occupation', 'relationship', 'race',\r\n                  'sex', 'native_country']\r\nadult_df_rev = adult_df_rev.drop(dummy_fields, axis = 1)<\/pre>\n<p>Using the above code snippets, we have created multiple categorical columns like <strong>&#8220;marital_cat&#8221;, &#8220;race_cat&#8221;<\/strong> etc. You can see the top 6 lines of the dataframe using <span class=\"lang:default decode:true crayon-inline\">adult_df_rev.head()<\/span><\/p>\n<p>By printing the\u00a0 <span class=\"lang:default decode:true crayon-inline \">adult_df_rev.head()<\/span>\u00a0 result. You will be able to see that all the columns should be reindexed. They are not in proper order. For reindexing the columns, you can use the code snippet provided below:<\/p>\n<pre class=\"lang:python decode:true\">adult_df_rev = adult_df_rev.reindex_axis(['age', 'workclass_cat', 'fnlwgt', 'education_cat',\r\n                                    'education_num', 'marital_cat', 'occupation_cat',\r\n                                    'relationship_cat', 'race_cat', 'sex_cat', 'capital_gain',\r\n                                    'capital_loss', 'hours_per_week', 'native_country_cat', \r\n                                    'income'], axis= 1)\r\n\r\nadult_df_rev.head(1)<\/pre>\n<p style=\"text-align: justify;\">The output to the above code snippet will show you that all the columns are reindexed properly. I have passed the list of column names as a\u00a0parameter and <strong>axis=1<\/strong> for reindexing the columns.<\/p>\n<h3>Standardization of Data<\/h3>\n<p style=\"text-align: justify;\">All the data values of our dataframe are numeric. Now, we need to convert them on a single scale. We can standardize the values. \u00a0We can use the below formula for standardization.<\/p>\n<p style=\"text-align: justify;\">$latex {x}_i = \\frac{{x}_i &#8211; mean(x)} {\\sigma(x)}$<\/p>\n<p>&nbsp;<\/p>\n<pre class=\"lang:python decode:true\" title=\"Standardization of Data\">num_features = ['age', 'workclass_cat', 'fnlwgt', 'education_cat', 'education_num',\r\n                'marital_cat', 'occupation_cat', 'relationship_cat', 'race_cat',\r\n                'sex_cat', 'capital_gain', 'capital_loss', 'hours_per_week',\r\n                'native_country_cat']\r\n\r\nscaled_features = {}\r\nfor each in num_features:\r\n    mean, std = adult_df_rev[each].mean(), adult_df_rev[each].std()\r\n    scaled_features[each] = [mean, std]\r\n    adult_df_rev.loc[:, each] = (adult_df_rev[each] - mean)\/std<\/pre>\n<p style=\"text-align: justify;\">We have converted our data values into standardized values. You can print and check the output of dataframe.<\/p>\n<h2>Data Slicing<\/h2>\n<p style=\"text-align: justify;\">Let&#8217;s split the data into training and test set. We can easily perform this step using sklearn&#8217;s train_test_split() method.<\/p>\n<pre class=\"lang:default decode:true \">features = adult_df_rev.values[:,:14]\r\ntarget = adult_df_rev.values[:,14]\r\nfeatures_train, features_test, target_train, target_test = train_test_split(features,\r\n                                                                            target, test_size = 0.33, random_state = 10)<\/pre>\n<p style=\"text-align: justify;\">Using above code snippet, we have divided the data into features and target set. The feature set consists of 14 columns i.e, predictor variables and target set consists of 1 column with class values.<\/p>\n<p style=\"text-align: justify;\">The <strong>features_train<\/strong> &amp; <strong>target_train<\/strong> consists of training data and the <strong>features_test<\/strong> &amp; <strong>target_test<\/strong> consists of testing data.<\/p>\n<h3>Gaussian Naive Bayes Implementation<\/h3>\n<p style=\"text-align: justify;\">After completing the data preprocessing.\u00a0it&#8217;s time to implement machine learning algorithm on it. We are going to use sklearn&#8217;s GaussianNB module.<\/p>\n<pre class=\"lang:default decode:true\">clf = GaussianNB()\r\nclf.fit(features_train, target_train)\r\ntarget_pred = clf.predict(features_test)<\/pre>\n<p style=\"text-align: justify;\">We have built a <strong>GaussianNB<\/strong>\u00a0classifier. The classifier is trained using training data. We can use <strong>fit()<\/strong> method for training it.\u00a0After building a classifier, our model is ready to make predictions. We can use <strong>predict()<\/strong> method with test set features as its parameters.<\/p>\n<h3>Accuracy of our Gaussian Naive Bayes model<\/h3>\n<p style=\"text-align: justify;\">It&#8217;s time to test the quality of our model. We have made some predictions. Let&#8217;s compare the model&#8217;s prediction with actual target values for the\u00a0test set.\u00a0By following this method, we are going to calculate the accuracy of our model.<\/p>\n<pre class=\"lang:default decode:true \">accuracy_score(target_test, target_pred, normalize = True)<\/pre>\n<h4>Script Output<\/h4>\n<pre class=\"lang:default decode:true\">0.80141447980643965<\/pre>\n<p>Awesome! Our model is\u00a0giving an <strong>accuracy of 80%<\/strong>. This is not bad with a simple implementation. You can create random test datasets and test the model to get know how well the trained Gaussian Naive Bayes model is performing.<\/p>\n<p>We can save the trained scikit-learn model with Python pickle. You can check out <a href=\"https:\/\/dataaspirant.com\/2017\/02\/13\/save-scikit-learn-models-with-python-pickle\/\" target=\"_blank\">how to save the trained scikit-learn model with Python Pickle<\/a>.<\/p>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>|\u00a0<span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span>\u00a0|\u00a0<span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span>\u00a0|\u00a0<span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0|\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions, then feel free to comment below.\u00a0 If you want me to write on one particular topic, then do tell it to me in the comments below.<\/p>\n<h4 style=\"text-align: justify;\">Related Data science Courses<\/h4>\n<ul>\n<li><a class=\"thirstylink\" title=\"Building Machine learning models with Python\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/data-science-machine-learning-python-hands-course-udemy\/\" target=\"_blank\" rel=\"nofollow noopener\">Building Machine learning models with Python<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"Python Machine learning hands on experience\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-z-hands-python-r-data-science-course-udemy\/\" target=\"_blank\" rel=\"nofollow noopener\">Python Machine learning hands on experience<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"Complete machine learning courses\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-courses-udemy\/\" target=\"_blank\" rel=\"nofollow noopener\">Complete machine learning courses<\/a><\/li>\n<\/ul>\n<\/div>\n<div class=\"CodeMirror-lines\">\n<p>&nbsp;<\/p>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Building Gaussian Naive Bayes Classifier in Python In this post,\u00a0we are going to implement the Naive Bayes classifier in Python using my favorite machine learning library scikit-learn. Next, we are going to use the trained Naive Bayes (supervised classification), model to predict the Census Income. As we discussed the Bayes theorem in naive Bayes classifier<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/gaussian-naive-bayes-classifier-implementation-python\/\">+ Read More<\/a><\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[6,8],"tags":[39,718,23,29],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1825"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=1825"}],"version-history":[{"count":1,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1825\/revisions"}],"predecessor-version":[{"id":11859,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1825\/revisions\/11859"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=1825"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=1825"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=1825"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}