{"id":630,"date":"2017-03-02T10:30:03","date_gmt":"2017-03-02T10:30:03","guid":{"rendered":"http:\/\/dataaspirant.com\/?p=630"},"modified":"2017-03-04T17:15:34","modified_gmt":"2017-03-04T17:15:34","slug":"how-logistic-regression-model-works","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/how-logistic-regression-model-works\/","title":{"rendered":"How the logistic regression model works"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><div id=\"attachment_632\" style=\"width: 810px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-632\" class=\"wp-image-632 size-full lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png\" alt=\"Logistic Regression Model\" width=\"800\" height=\"400\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-300x150.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-768x384.png 768w\" data-sizes=\"(max-width: 800px) 100vw, 800px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 800px; --smush-placeholder-aspect-ratio: 800\/400;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-632\" class=\"wp-image-632 size-full\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png\" alt=\"Logistic Regression Model\" width=\"800\" height=\"400\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-300x150.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-768x384.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/noscript><p id=\"caption-attachment-632\" class=\"wp-caption-text\">Logistic Regression Model<\/p><\/div>\n<h2>How the Logistic Regression Model Works in Machine Learning<\/h2>\n<p style=\"text-align: justify;\">In this\u00a0article, we are going to learn how the logistic regression model works in machine learning. The logistic regression model is one member of the <a href=\"https:\/\/dataaspirant.com\/2014\/09\/19\/supervised-and-unsupervised-learning\/\" target=\"_blank\">supervised classification algorithm<\/a>\u00a0family. The building block concepts of logistic regression can be helpful in <strong>deep learning<\/strong> while building the neural networks.<\/p>\n<p style=\"text-align: justify;\"><!--more--><\/p>\n<p style=\"text-align: justify;\">Logistic regression classifier is more like a <a href=\"https:\/\/dataaspirant.com\/2014\/10\/02\/linear-regression\/\" target=\"_blank\"><strong>linear classifier<\/strong><\/a> which uses the calculated logits (score ) to predict the target class. If you are not familiar with the concepts of the\u00a0logits, don&#8217;t frighten. We are going to learn each and every block of logistic regression by the end of this post.<\/p>\n<p style=\"text-align: justify;\">Before we begin, let&#8217;s check out the table of contents.<\/p>\n<h4>Table of Contents<\/h4>\n<ul>\n<li>What is logistic regression?<\/li>\n<li>Dependent and Independent variables?<\/li>\n<li>Logistic regression examples.<\/li>\n<li>How the logistic regression classifier works.\n<ul>\n<li>Binary classification with logistic regression model.<\/li>\n<\/ul>\n<\/li>\n<li>What is Softmax function?<\/li>\n<li>The special cases of softmax function.<\/li>\n<li>Implementing the softmax function in Python.<\/li>\n<\/ul>\n<p>So let&#8217;s begin.<\/p>\n<span class='bctt-click-to-tweet'><span class='bctt-ctt-text'><a href='https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fdataaspirant.com%2F%3Fp%3D630&#038;text=How%20the%20Logistic%20Regression%20Model%20Works%20in%20Machine%20Learning.&#038;via=dataaspirant&#038;related=dataaspirant' target='_blank'rel=\"noopener noreferrer\">How the Logistic Regression Model Works in Machine Learning. <\/a><\/span><a href='https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fdataaspirant.com%2F%3Fp%3D630&#038;text=How%20the%20Logistic%20Regression%20Model%20Works%20in%20Machine%20Learning.&#038;via=dataaspirant&#038;related=dataaspirant' target='_blank' class='bctt-ctt-btn'rel=\"noopener noreferrer\">Click To Tweet<\/a><\/span>\n<h3>What is logistic regression?<\/h3>\n<p>Below is the most accurate and well-defined definition of logistic regression from Wikipedia.<\/p>\n<blockquote><p><em>&#8220;Logistic regression measures the relationship between the categorical dependent variable and one or more independent variables by estimating probabilities using a <a title=\"Logistic function\" href=\"https:\/\/en.wikipedia.org\/wiki\/Logistic_function\" target=\"_blank\">logistic function<\/a>&#8221;\u00a0<\/em>(<a href=\"https:\/\/en.wikipedia.org\/wiki\/Logistic_regression\" target=\"_blank\">Wikipedia<\/a>)<\/p><\/blockquote>\n<p style=\"text-align: justify;\">Let&#8217;s understand the above logistic regression model definition word by word. What logistic regression model will do is, It uses a black box function to understand the relation between the categorical <strong>dependent variable<\/strong> and the <strong>independent variables<\/strong>. This black box function is popularly known as the <strong>Softmax funciton<\/strong>.<\/p>\n<h3>Dependent and Independent Variables<\/h3>\n<p style=\"text-align: justify;\">The dependent and the independent variables are the same which we were discussed in the <a href=\"https:\/\/dataaspirant.com\/2017\/02\/15\/simple-linear-regression-python-without-any-machine-learning-libraries\/\" target=\"_blank\">building simple linear regression model<\/a>. Just to give you a glance. The dependent variable is the target class variable we are going to predict. However, the independent variables are the features or attributes we are going to use to predict the target class.<\/p>\n<p style=\"text-align: justify;\">Suppose the shop owner would like to predict the customer who entered into the shop will buy the <strong>Macbook or Not<\/strong>. To predict whether the customer will buy the MacBook or not. The shop owner will observe the customer features like.<\/p>\n<ul>\n<li><strong>Gender:<\/strong>\n<ul>\n<li>Probabilities wise male will high chances of purchasing a MacBook than females.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Age:<\/strong>\n<ul>\n<li>Kids won&#8217;t purchase MacBook.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">The shop owner will use the above, similar kind of features to predict the <strong>likelihood occurrence<\/strong> of the event \u00a0(Will buy the Macbook or not.)<\/p>\n<p style=\"text-align: justify;\">In the mathematical side, the logistic regression model will pass the likelihood occurrences through the <strong>logistic function<\/strong> to predict the corresponding target class. This popular\u00a0logistic function is the Softmax function. We are going to learn about the softmax function in the coming sections of this post.<\/p>\n<p style=\"text-align: justify;\">Before that. Let&#8217;s quickly see few examples to understand the sentence <strong>likelihood occurrence of an event<\/strong>.<\/p>\n<h3>Examples of likelihood occurrence of an event<\/h3>\n<ul>\n<li>How likely a customer will buy iPod having iPhone in his\/her pocket.<\/li>\n<li>How likely Argentina team will win when <a href=\"https:\/\/en.wikipedia.org\/wiki\/Lionel_Messi\" target=\"_blank\">Lionel Andr\u00e9s Messi<\/a> in rest.<\/li>\n<li>What is the probability to get into best university\u00a0by scoring decent marks in mathematics, physics?<\/li>\n<li>What is the probability to get a kiss from your girlfriend\u00a0when you gifted her favorite\u00a0dress on behalf of your birthday?<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Hope\u00a0the above examples gives you the better idea about the sentence <strong>predict the likelihood occurrence of an event.<\/strong><\/p>\n<p style=\"text-align: justify;\">Before drive into the underline\u00a0mathematical concept of logistic regression. let&#8217;s brush up the logistic regression understanding level with an example.<\/p>\n<h3>Logistic Regression Model Example<\/h3>\n<div id=\"attachment_632\" style=\"width: 810px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-632\" class=\"size-full wp-image-632 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png\" alt=\"logistic Regression Example\" width=\"800\" height=\"400\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-300x150.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-768x384.png 768w\" data-sizes=\"(max-width: 800px) 100vw, 800px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 800px; --smush-placeholder-aspect-ratio: 800\/400;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-632\" class=\"size-full wp-image-632\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png\" alt=\"logistic Regression Example\" width=\"800\" height=\"400\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-300x150.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2016\/04\/logisticregression-768x384.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/noscript><p id=\"caption-attachment-632\" class=\"wp-caption-text\"><strong>logistic Regression Example<\/strong><\/p><\/div>\n<p style=\"text-align: justify;\">Suppose <strong>HPenguin\u00a0<\/strong>wants to know, how likely it will be happy based on its daily activities. If the penguin wants to build a logistic regression model to predict it\u00a0happiness based on its daily activities. The penguin needs both the happy and sad activities. In <a href=\"https:\/\/dataaspirant.com\/category\/machine-learning-2\/\" target=\"_blank\">machine learning<\/a>\u00a0terminology these activities are known as the Input parameters ( <strong>features<\/strong> ).<\/p>\n<p style=\"text-align: justify;\">So let&#8217;s create a table which contains penguin activities and the result of that activity like <strong>happy<\/strong> or <strong>sad<\/strong>.<\/p>\n<p>&nbsp;<\/p>\n<table class=\"tg\">\n<tbody>\n<tr>\n<th class=\"tg-baqh\">No.<\/th>\n<th class=\"tg-yw4l\">Penguin Activity<\/th>\n<th class=\"tg-yw4l\">Penguin Activity Description<\/th>\n<th class=\"tg-yw4l\">How Penguin felt ( Target )<\/th>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">1<\/td>\n<td class=\"tg-yw4l\">X1<\/td>\n<td class=\"tg-yw4l\">Eating squids<\/td>\n<td class=\"tg-yw4l\">Happy<\/td>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">2<\/td>\n<td class=\"tg-yw4l\">X2<\/td>\n<td class=\"tg-yw4l\">Eating small Fishes<\/td>\n<td class=\"tg-yw4l\">Happy<\/td>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">3<\/td>\n<td class=\"tg-yw4l\">X3<\/td>\n<td class=\"tg-yw4l\">Hit by other Penguin<\/td>\n<td class=\"tg-yw4l\">Sad<\/td>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">4<\/td>\n<td class=\"tg-yw4l\">X4<\/td>\n<td class=\"tg-yw4l\">Eating Crabs<\/td>\n<td class=\"tg-yw4l\">Sad<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify;\">Penguin is going to use the above activities ( features ) to train the logistic regression model. Later the trained logistic regression model will predict how the penguin is feeling for the new penguin activities.<\/p>\n<p>As it&#8217;s not possible to use the above categorical data table to build the logistic regression. \u00a0The above activities data table needs to convert into activities score, weights, and the corresponding target.<\/p>\n<h4>Penguin Activity Data table<\/h4>\n<table class=\"tg\">\n<tbody>\n<tr>\n<th class=\"tg-9hbo\">No.<\/th>\n<th class=\"tg-9hbo\">Penguin Activity<\/th>\n<th class=\"tg-9hbo\">Activity Score<\/th>\n<th class=\"tg-9hbo\">Weights<\/th>\n<th class=\"tg-9hbo\">Target<\/th>\n<th class=\"tg-9hbo\">Target Description<\/th>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">1<\/td>\n<td class=\"tg-yw4l\">X1<\/td>\n<td class=\"tg-yw4l\">6<\/td>\n<td class=\"tg-yw4l\">0.6<\/td>\n<td class=\"tg-yw4l\">1<\/td>\n<td class=\"tg-yw4l\">Happy<\/td>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">2<\/td>\n<td class=\"tg-yw4l\">X2<\/td>\n<td class=\"tg-yw4l\">3<\/td>\n<td class=\"tg-yw4l\">0.4<\/td>\n<td class=\"tg-yw4l\">1<\/td>\n<td class=\"tg-yw4l\">Happy<\/td>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">3<\/td>\n<td class=\"tg-yw4l\">X3<\/td>\n<td class=\"tg-yw4l\">7<\/td>\n<td class=\"tg-yw4l\">-0.7<\/td>\n<td class=\"tg-yw4l\">0<\/td>\n<td class=\"tg-yw4l\">Sad<\/td>\n<\/tr>\n<tr>\n<td class=\"tg-yw4l\">4<\/td>\n<td class=\"tg-yw4l\">X4<\/td>\n<td class=\"tg-yw4l\">3<\/td>\n<td class=\"tg-yw4l\">-0.3<\/td>\n<td class=\"tg-yw4l\">0<\/td>\n<td class=\"tg-28r4\">Sad<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The updated dataset looks like this. Before we drive further let&#8217;s understand more about the above data table.<\/p>\n<ul>\n<li>\n<h4>Penguin Activity:<\/h4>\n<ul>\n<li>The activities penguin do daily like eating small fishes, eating crabs .. etc<\/li>\n<\/ul>\n<\/li>\n<li>\n<h4>Activity Score:<\/h4>\n<ul>\n<li>The activity score is more like the numerical equivalent to the penguin activity. For eating\u00a0squids activity, the corresponding activity score is 6 and likewise, for other activities the scores are 3, 7, 3.<\/li>\n<\/ul>\n<\/li>\n<li>\n<h4>Weights:<\/h4>\n<ul>\n<li>The Weights more like the weightages corresponding to the particular target.<\/li>\n<li>Suppose for the activity X1 we have the weight as 0.6.<\/li>\n<li>It means to say if the penguin performs the activity X1 the model is 60% confident to say the penguin will be happy.<\/li>\n<li>If you observe the weights for the\u00a0target class <strong>happy<\/strong> are positive, and the weights for the target class <strong>sad <\/strong>are negative.<\/li>\n<li>This is because the problem we are addressing a binary classification. Will talk about the binary classification in the next sections of this post.<\/li>\n<\/ul>\n<\/li>\n<li>\n<h4>Target:<\/h4>\n<ul>\n<li>The target is just the binary values. The value 1 represents the target <strong>Happy<\/strong>, and the value 0 represents the target<strong> Sad<\/strong>.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify;\">Now we know the activity score for each activity and the corresponding weights. To predict how the\u00a0penguin will feel given the activity we just need to <strong>multiply the activity score and the corresponding weight<\/strong> to get the score. The\u00a0calculated score is also known as the <strong>logits<\/strong> which we talked earlier\u00a0in the post.<\/p>\n<p style=\"text-align: justify;\">The logit (Score) will pass into the softmax function to get the probability\u00a0for <strong>each target class<\/strong>. In our case, if we pass the logit through the softmax function will get the probability for the target <strong>happy class<\/strong> and for the target<strong> sad class<\/strong>. Later we can consider the target class with<strong> high probability<\/strong> as the predicted target class for the\u00a0given activity.<\/p>\n<p style=\"text-align: justify;\">In fact, we can predict whether the Penguin is feeling happy or sad with the calculated<strong> logits (Score ) in this case<\/strong>. As we were given the positive weights for the target class happy and the negative weights for the target class sad. We can say If the Logit is <strong>greater than 0<\/strong> the target class is happy and if the logit is <strong>less than \u00a00<\/strong> the target class is sad.<\/p>\n<div id=\"attachment_2376\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2376\" class=\"size-full wp-image-2376 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/logistic_regression_model.jpg\" alt=\"logistic regression model for binary classification\" width=\"700\" height=\"400\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/logistic_regression_model.jpg 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/logistic_regression_model-300x171.jpg 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/400;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2376\" class=\"size-full wp-image-2376\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/logistic_regression_model.jpg\" alt=\"logistic regression model for binary classification\" width=\"700\" height=\"400\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/logistic_regression_model.jpg 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/logistic_regression_model-300x171.jpg 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2376\" class=\"wp-caption-text\">Logistic regression model for binary classification<\/p><\/div>\n<h3>Binary classification with Logistic Regression model<\/h3>\n<p style=\"text-align: justify;\">In the Penguin example, we pre-assigned the <strong>activity scores and the weights<\/strong> for the logistic regression model. This won&#8217;t be the simple while modeling the logistic regression model for real word problems. The weights will be calculated over the training data set. Using the calculated the weights the Logits will be computed. Till here the model is similar to the <a href=\"https:\/\/dataaspirant.com\/2014\/10\/02\/linear-regression\/\" target=\"_blank\">linear regression model<\/a>.<\/p>\n<p style=\"text-align: justify;\"><strong>Note:<\/strong> The Logits in the image were just for example, and <strong>not the calculated<\/strong> logits from the penguin example<\/p>\n<p>The calculated Logits (score) for the linear regression model will pass through the softmax function. The softmax function will return the probabilities for each target class. The high probability target class will be the predicted target class.<\/p>\n<p>The target classes \u00a0In the Penguin example, are having two target classes (Happy and Sad). If we are using the logistic regression model for predicting the binary targets like yes or no, 1 or 0. which is known as the Binary classification.<\/p>\n<p>Till now we talk about the <strong>softmax function<\/strong> as a black box which takes the calculated scores and returns the probabilities. Now let&#8217;s learn how the softmax function calculates the probabilities. Next, we are going to implement the simple softmax function to calculate the probabilities for given Logits (Scores)<\/p>\n<h3>What is Softmax function?<\/h3>\n<div id=\"attachment_2380\" style=\"width: 810px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2380\" class=\"size-full wp-image-2380 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function.png\" alt=\"softmax function\" width=\"800\" height=\"169\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function-300x63.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function-768x162.png 768w\" data-sizes=\"(max-width: 800px) 100vw, 800px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 800px; --smush-placeholder-aspect-ratio: 800\/169;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2380\" class=\"size-full wp-image-2380\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function.png\" alt=\"softmax function\" width=\"800\" height=\"169\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function.png 800w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function-300x63.png 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/03\/softmax_function-768x162.png 768w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/noscript><p id=\"caption-attachment-2380\" class=\"wp-caption-text\"><strong>Softmax function<\/strong><\/p><\/div>\n<p>Softmax function is the popular function to calculate the <strong>probabilities<\/strong> of the events. The other mathematical advantages of using the softmax function are the output range. \u00a0Softmax function output values are always in the<strong> range of (0, 1)<\/strong>. The sum of the output values will always <strong>equal to the 1<\/strong>. The Softmax is also known as the <strong>normalized exponential<\/strong> function.<\/p>\n<p>The above is the softmax formula. Which takes each value\u00a0(Logits) and find the probability. The numerator the e-power values of the Logit and the denominator calculates the sum of the e-power\u00a0values of all the Logits.<\/p>\n<h4>Softmax function used in:<\/h4>\n<ul>\n<li><a href=\"https:\/\/dataaspirant.com\/2017\/02\/06\/naive-bayes-classifier-machine-learning\/\" target=\"_blank\">Naive Bayes Classifier<\/a><\/li>\n<li>Multinomial Logistic Classifier<\/li>\n<li>Deep Learning (While building Neural networks)<\/li>\n<\/ul>\n<p>Before we implementing the softmax function, Let&#8217;s study the special cases of the Softmax function inputs.<\/p>\n<h3>The special cases of softmax function input<\/h3>\n<p>The two special cases we need to consider about the Softmax function output, If we do the below modifications to the Softmax function inputs.<\/p>\n<ul>\n<li>Multiplying the Softmax function inputs (Multiplying the Logits with any value)<\/li>\n<li>Dividing the Softmax function inputs (Dividing the Logits with any value)<\/li>\n<\/ul>\n<h4>Multiplying the Softmax function inputs:<\/h4>\n<p style=\"text-align: justify;\">If we multiply the Softmax function inputs, the inputs values will become large. So the logistic regression will be more confident (High Probability value) about the predicted target class.<\/p>\n<h4>Dividing the Softmax function inputs:<\/h4>\n<p style=\"text-align: justify;\">If we divide the Softmax function inputs, the inputs values will become small. So the Logistic regression model will be not confident (Less Probability value) of the predicted target class.<\/p>\n<p style=\"text-align: justify;\">Enough of the theoretical\u00a0concept of the Softmax function. Let&#8217;s do the fun part (Coding).<\/p>\n<h3>Implementing the softmax function in Python<\/h3>\n<p>Let&#8217;s implement a softmax function which takes the logits in list or array and returns the softmax function outputs in a list.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Softmax function\"># Required Python Packages\r\nimport numpy as np\r\n\r\n\r\ndef softmax(scores):\r\n    \"\"\"\r\n    Calculate the softmax for the given scores\r\n    :param scores:\r\n    :return:\r\n    \"\"\"\r\n    return np.exp(scores) \/ np.sum(np.exp(scores), axis=0)\r\n\r\nscores = [8, 5, 2]\r\n\r\nif __name__ == \"__main__\":\r\n    logits = [8, 5, 2]\r\n    print \"Softmax :: \", softmax(logits)<\/pre>\n<p>To implement the softmax function we just replicated the Softmax formula.<\/p>\n<ul>\n<li>The input to the softmax function is the logits in a list or array.<\/li>\n<li>The numerator computes the exponential value of each Logit in the array.<\/li>\n<li>The denominator calculates the sum of all exponential values.<\/li>\n<li>Finally, we return the ratio of the numerator and the denominator values.<\/li>\n<\/ul>\n<h4>Script Output:<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Softmax function output\">Softmax ::  [ 0.95033021  0.04731416  0.00235563]\r\n<\/pre>\n<h3>Summary<\/h3>\n<p>In this post, we learned about the logistic regression model with a toy kind of example. Further, we learned about the softmax function. Finally, we implemented the simple softmax function with takes the logits as input and returns the probabilities as the outputs.<\/p>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>|\u00a0<span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span>\u00a0|\u00a0<span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span>\u00a0|\u00a0<span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0|\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions, then feel free to comment below.\u00a0 If you want me to write on one particular topic, then do tell it to me in the comments below.<\/p>\n<h4 style=\"text-align: justify;\">Related Data science Courses<\/h4>\n<ul>\n<li><a class=\"thirstylink\" title=\"Building Machine learning models with Python\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/data-science-machine-learning-python-hands-course-udemy\/\" rel=\"nofollow\" target=\"_blank\">Building Machine learning models with Python<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"Python Machine learning hands on experience\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-z-hands-python-r-data-science-course-udemy\/\" rel=\"nofollow\" target=\"_blank\">Python Machine learning hands on experience<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"Complete machine learning courses\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-courses-udemy\/\" rel=\"nofollow\" target=\"_blank\">Complete machine learning courses<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>How the Logistic Regression Model Works in Machine Learning In this\u00a0article, we are going to learn how the logistic regression model works in machine learning. The logistic regression model is one member of the supervised classification algorithm\u00a0family. The building block concepts of logistic regression can be helpful in deep learning while building the neural networks.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[6],"tags":[39,1016],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/630"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=630"}],"version-history":[{"count":0,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/630\/revisions"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=630"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=630"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=630"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}