{"id":2790,"date":"2017-04-15T13:56:24","date_gmt":"2017-04-15T13:56:24","guid":{"rendered":"http:\/\/dataaspirant.com\/?p=2790"},"modified":"2017-04-20T11:35:04","modified_gmt":"2017-04-20T11:35:04","slug":"implement-logistic-regression-model-python-binary-classification","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/implement-logistic-regression-model-python-binary-classification\/","title":{"rendered":"How to implement logistic regression model in python for binary classification"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><div id=\"attachment_2798\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2798\" class=\"size-full wp-image-2798 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Logistic-Regression-Python.jpg\" alt=\"Logistic Regression Python\" width=\"700\" height=\"400\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Logistic-Regression-Python.jpg 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Logistic-Regression-Python-300x171.jpg 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/400;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2798\" class=\"size-full wp-image-2798\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Logistic-Regression-Python.jpg\" alt=\"Logistic Regression Python\" width=\"700\" height=\"400\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Logistic-Regression-Python.jpg 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Logistic-Regression-Python-300x171.jpg 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2798\" class=\"wp-caption-text\"><strong>Logistic Regression Python<\/strong><\/p><\/div>\n<p style=\"text-align: justify;\">In the last few articles, we talked about different classification algorithms. For every <a href=\"https:\/\/dataaspirant.com\/tag\/classification-algorithms\/\" target=\"_blank\">classification algorithm<\/a>, we learn the background concepts of the algorithm and in the followed article we used the background concepts of the algorithm to build the classification model. Later we used the mode to perform the\u00a0<a href=\"https:\/\/dataaspirant.com\/2014\/09\/27\/classification-and-prediction\/\" target=\"_blank\">regression or classification<\/a> tasks.<\/p>\n<p style=\"text-align: justify;\">Likewise in this article, we are going to implement the logistic regression model in python to perform the binary classification task. In this, we are mainly concentrating\u00a0on the implementation of logistic regression in python, as the background concepts explained in <a href=\"https:\/\/dataaspirant.com\/2017\/03\/14\/multinomial-logistic-regression-model-works-machine-learning\/\" target=\"_blank\">how the logistic regression model<\/a> works article.<\/p>\n<p style=\"text-align: justify;\">Take a cup of tea\/coffee and check out the below prerequisite\u00a0concepts before we drive further.<\/p>\n<ul>\n<li style=\"text-align: justify;\">\n<blockquote><p><a href=\"https:\/\/dataaspirant.com\/2017\/03\/02\/how-logistic-regression-model-works\/\" target=\"_blank\">How the logistic regression works.<\/a><\/p><\/blockquote>\n<\/li>\n<li style=\"text-align: justify;\">\n<blockquote><p><a href=\"https:\/\/dataaspirant.com\/2017\/03\/14\/multinomial-logistic-regression-model-works-machine-learning\/\" target=\"_blank\">How multinomial logistic regression algorithm works.<\/a><\/p><\/blockquote>\n<\/li>\n<li style=\"text-align: justify;\">\n<blockquote><p><a href=\"https:\/\/dataaspirant.com\/2017\/03\/07\/difference-between-softmax-function-and-sigmoid-function\/\" target=\"_blank\">Key differences between Softmax and sigmoid functions.<\/a><\/p><\/blockquote>\n<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Now let&#8217;s view the concepts we are going to learn by the end of this article.<\/p>\n<h3>Table of contents<\/h3>\n<ul>\n<li>What is binary classification<\/li>\n<li>Logistic regression introduction<\/li>\n<li>Building logistic regression model in python\n<ul>\n<li>Binary classification problem<\/li>\n<li>Dataset description<\/li>\n<li>Data creation for modeling and testing\n<ul>\n<li>Selecting the features<\/li>\n<li>Split the data into train and test dataset<\/li>\n<\/ul>\n<\/li>\n<li>Understanding the training data<\/li>\n<li>Implementing the logistic regression model in python with scikit-learn<\/li>\n<li>Logistic regression model \u00a0accuracy calculation<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<span class='bctt-click-to-tweet'><span class='bctt-ctt-text'><a href='https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fdataaspirant.com%2F%3Fp%3D2790&#038;text=Building%20Logistic%20regression%20classifier%20in%20Python&#038;via=dataaspirant&#038;related=dataaspirant' target='_blank'rel=\"noopener noreferrer\">Building Logistic regression classifier in Python <\/a><\/span><a href='https:\/\/twitter.com\/intent\/tweet?url=https%3A%2F%2Fdataaspirant.com%2F%3Fp%3D2790&#038;text=Building%20Logistic%20regression%20classifier%20in%20Python&#038;via=dataaspirant&#038;related=dataaspirant' target='_blank' class='bctt-ctt-btn'rel=\"noopener noreferrer\">Click To Tweet<\/a><\/span>\n<h2>What is binary classification<\/h2>\n<p style=\"text-align: justify;\">Binary classification is performing the task of classifying the binary targets with the use of <a href=\"https:\/\/dataaspirant.com\/2014\/09\/19\/supervised-and-unsupervised-learning\/\" target=\"_blank\">supervised classification algorithms<\/a>. The binary target means having only 2 targets values\/classes. To get the clear picture about the binary classification lets looks at the below binary classification problems.<\/p>\n<ul>\n<li style=\"text-align: justify;\">Identifying the image as a cat or not.\n<ul>\n<li style=\"text-align: justify;\"><strong>Targets:<\/strong> cat or not a cat<\/li>\n<\/ul>\n<\/li>\n<li style=\"text-align: justify;\">Predicting to whom the voter will vote\u00a0Bill Clinton or\u00a0Bob Dole\n<ul>\n<li style=\"text-align: justify;\"><strong>Targets:<\/strong> Bill Clinton or Bod Dole<\/li>\n<\/ul>\n<\/li>\n<li style=\"text-align: justify;\">Forecasting will it rain tomorrow.\n<ul>\n<li style=\"text-align: justify;\"><strong>Targets:<\/strong> Will rain or sunny day<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Hope with the above classification problems you are having the clear understanding on the binary classification problems.<\/p>\n<h2>Logistic regression introduction<\/h2>\n<p style=\"text-align: justify;\">The logistic regression algorithm\u00a0is the simplest classification algorithm used for the binary classification task. Which can also be used for solving the multi-classification problems. In summarizing way of saying logistic regression model will take the feature values and calculates the probabilities using the <a href=\"https:\/\/dataaspirant.com\/2017\/03\/07\/difference-between-softmax-function-and-sigmoid-function\/\" target=\"_blank\">sigmoid or softmax <\/a>functions.<\/p>\n<p style=\"text-align: justify;\">The sigmoid function used for binary classification problems and Softmax function used of multi-classification problems.<\/p>\n<p style=\"text-align: justify;\">Later the calculated probabilities used to find the target class. In general, the high probability class treated as the final target class.<\/p>\n<p style=\"text-align: justify;\">The above explanation is indeed as you already know the logistic regression algorithm. If you new to the logistic regression algorithm please check out <a href=\"https:\/\/dataaspirant.com\/2017\/03\/02\/how-logistic-regression-model-works\/\" target=\"_blank\">how the logistic regression algorithm<\/a> works before you continue this article.<\/p>\n<h2>Building logistic regression model in python<\/h2>\n<p style=\"text-align: justify;\">To build the logistic regression model in python we are going to use the <a href=\"http:\/\/scikit-learn.org\/\" target=\"_blank\">Scikit-learn<\/a> package. We are going to follow the below workflow for implementing the logistic regression model.<\/p>\n<ul>\n<li>Load the data set.<\/li>\n<li>Understanding the data.<\/li>\n<li>Split the data into training and test dataset.<\/li>\n<li>Use the training dataset to model the logistic regression model.<\/li>\n<li>Calculate the accuracy of the trained model on the\u00a0training dataset.<\/li>\n<li>Calculate the accuracy of the model on the test dataset.<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">We are going to build the logistic regression model in the above workflow to address the binary classification problem. Let&#8217;s look into the problem we are going to solve.<\/p>\n<h3>Binary classification problem<\/h3>\n<div id=\"attachment_2801\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2801\" class=\"size-full wp-image-2801 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model.jpg\" alt=\"Predicting Election Results with Logistic Regression model\" width=\"700\" height=\"400\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model.jpg 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model-300x171.jpg 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/400;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2801\" class=\"size-full wp-image-2801\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model.jpg\" alt=\"Predicting Election Results with Logistic Regression model\" width=\"700\" height=\"400\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model.jpg 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model-300x171.jpg 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2801\" class=\"wp-caption-text\">Predicting Election Results with Logistic Regression model<\/p><\/div>\n<p>We are going to build the logistic regression model to predict, for whom the voter will vote. Given the voter details.<\/p>\n<ul>\n<li>Will the voter will vote for Bill Clinton?<\/li>\n<li>Will the voter will vote for Bobe Dole?<\/li>\n<\/ul>\n<h3>Dataset description<\/h3>\n<p>The dataset we are going to use is the 1996 United States President election dataset. In a moment we are going to look into all the features in the dataset. Before that, let me give you the quick summary about this election.<\/p>\n<h3>The United States 1996 President Election Summary<\/h3>\n<ul>\n<li>This 53rd united states president elections held on November 5, 1996.<\/li>\n<li>Nominees:\n<ul>\n<li>Bill Clinton<\/li>\n<li>Boby Dole<\/li>\n<li>Ross Perot<\/li>\n<\/ul>\n<\/li>\n<li>With <strong>49.2 %<\/strong> vote\u00a0percentage Bill Clinton won the Elections.<\/li>\n<\/ul>\n<p>Love to read more about the election? Then check out the few details about the election in wiki <a href=\"https:\/\/en.wikipedia.org\/wiki\/United_States_presidential_election,_1996\" target=\"_blank\">United States President Election<\/a>\u00a0article.<\/p>\n<p>Before we begin the modeling let&#8217;s import the <a href=\"https:\/\/dataaspirant.com\/2014\/11\/01\/python-packages-for-datamining\/\" target=\"_blank\">required python packages<\/a>.<\/p>\n<pre class=\"lang:python decode:true\" title=\"Required Python Packages\"># Required Python Packages\r\nimport pandas as pd\r\nimport numpy as np\r\nimport plotly.plotly as py\r\nimport plotly.graph_objs as go\r\npy.sign_in('YOUR_PLOTLY_USER_NAME', 'API_KEY')\r\n\r\nfrom sklearn.cross_validation import train_test_split\r\nfrom sklearn.linear_model import LogisticRegression\r\nfrom sklearn import metrics<\/pre>\n<ul>\n<li style=\"text-align: justify;\"><strong>Pandas package<\/strong> is required for data analysis. In the process of modeling logistic regression classifier, first we are going to load the dataset (CSV format) into pandas data frame and then we play around with the loaded dataset.<\/li>\n<li style=\"text-align: justify;\"><strong>Numpy package<\/strong> is for performing the numerical calculation.<\/li>\n<li style=\"text-align: justify;\"><strong>Plotly package<\/strong> for visualizing the data set for better understanding.\n<ul>\n<li style=\"text-align: justify;\">We need to sign_in with your\u00a0plotly credential to use this package. You can find the credential in your plotly account after you create an account.<\/li>\n<\/ul>\n<\/li>\n<li><strong>Sklearn package<\/strong> is for modeling the machine learning algorithms.\n<ul>\n<li><strong>train_test_split<\/strong> method to split the dataset into the train and test dataset.<\/li>\n<li><strong>LogisticRegression<\/strong> method for modeling the logistic regression classifier.<\/li>\n<li><strong>metrics<\/strong> method for calculating the accuracy of the trained classifiers.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p style=\"text-align: justify;\">Now let&#8217;s load the data set and look into all the features available to model the logistic regression model in python. You can download the data set from our\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/tree\/master\/Logistic_Regression\/Logistic_Binary_Classification\" target=\"_blank\">GitHub<\/a>.<\/p>\n<h3>Load the dataset<\/h3>\n<pre class=\"lang:python decode:true \" title=\"Loading the data set\"># Files\r\nDATA_SET_PATH = \"..\/Inputs\/anes_dataset.csv\"\r\n\r\n\r\ndef main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n    print \"Number of Observations :: \", len(dataset)\r\n\r\n    \r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<ul>\n<li>Created the <strong>main<\/strong> function and loading the elections dataset into pandas dataframe. As the dataset in CSV format, we are calling the pandas <strong>read_csv<\/strong> function with dataset path as a parameter.<\/li>\n<li>To know the number of observations (rows) in the dataset, we are calling the python <strong>len()<\/strong> function with the loaded dataset.<\/li>\n<\/ul>\n<h4>Script Output:<\/h4>\n<pre class=\"lang:python decode:true \" title=\"Script Output\">Number of Observations:: 944<\/pre>\n<p>From the script output, the number observation in the dataset are 944. We are going to play with the observation to model the logistic regression model \ud83d\ude42<\/p>\n<p>As we already know the dataset size, now lets chek out the few observation to know about the features in the dataset.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Top observation of the loaded dataset\">def main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n    print \"Number of Observations :: \", len(dataset)\r\n\r\n    # Get the first observation\r\n    print dataset.head()\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p>We can use the pandas <strong>head<\/strong> method to get the top observations of the loaded dataset.<\/p>\n<h4>Script Output:<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Script Output\">popul  TVnews  selfLR  ClinLR  DoleLR  PID  age  educ  income  vote\r\n  0       7       7       1       6    6   36     3       1     1\r\n 190       1       3       3       5    1   20     4       1     0\r\n  31       7       2       2       6    1   24     6       1     0\r\n  83       4       3       4       5    1   28     6       1     0\r\n 640       7       5       6       4    0   68     6       1     0<\/pre>\n<p style=\"text-align: justify;\">let&#8217;s write a function to get the header names of the given dataset. Later we store all the header names, which can be used in modeling the logistic regression.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Function to get dataset header names\">def dataset_headers(dataset):\r\n    \"\"\"\r\n    To get the dataset header names\r\n    :param dataset: loaded dataset into pandas DataFrame\r\n    :return: list of header names\r\n    \"\"\"\r\n    return list(dataset.columns.values)<\/pre>\n<p style=\"text-align: justify;\">This <strong>dataset_headers<\/strong>\u00a0function takes the dataset (loaded ) and returns the <strong>header names<\/strong> in the loaded dataset. As the dataset_headers function excepts the loaded dataset, we are going to call this function inside the main function we wrote earlier.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Calling the header_names funciton from main\">def main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n\r\n    headers = dataset_headers(dataset)\r\n    print \"Data set headers :: {headers}\".format(headers=headers)\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p>&nbsp;<\/p>\n<h4>Script Output:<\/h4>\n<pre class=\"lang:python decode:true \" title=\"Script Output\">Data set headers :: ['popul', 'TVnews', 'selfLR', 'ClinLR', 'DoleLR', 'PID', 'age', 'educ', 'income', 'vote']<\/pre>\n<p>Now let&#8217;s discuss each header (feature).<\/p>\n<ol>\n<li>popul:\n<ul>\n<li>Means the population in the census place in 1000<\/li>\n<\/ul>\n<\/li>\n<li>TVnews:\n<ul>\n<li>The number of time the voter views the Tv news in a week.<\/li>\n<\/ul>\n<\/li>\n<li>selfLR\n<ul>\n<li><span class=\"tx\">Is the person&#8217;s self<\/span><span class=\"tx\">&#8211;<\/span><span class=\"tx\">reported political learnings from left to right.\u00a0<\/span><\/li>\n<\/ul>\n<\/li>\n<li>ClinLR\n<ul>\n<li><span class=\"tx\">Is the person&#8217;s impression on Bill Clinton&#8217;s Political learning <\/span><br \/>\n<span class=\"tx\">from left to right<\/span><\/li>\n<\/ul>\n<\/li>\n<li>DoleLR\n<ul>\n<li><span class=\"tx\">Is the person impression of Bob Dole&#8217;s Political learnings from <\/span><span class=\"tx\">left to right.<\/span><\/li>\n<\/ul>\n<\/li>\n<li>PID\n<ul>\n<li>Party Identification of the person.<\/li>\n<li>If the PID is\n<ul>\n<li>0 means the s<span class=\"tx\">Strong Democrat, <\/span><\/li>\n<li><span class=\"tx\">1 means Week <\/span><span class=\"tx\">democrat, <\/span><\/li>\n<li><span class=\"tx\">2 means Independent democrat likewise<\/span><\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<li>age\n<ul>\n<li>Age of the voter.<\/li>\n<\/ul>\n<\/li>\n<li>educ\n<ul>\n<li>Education qualification of the voter.<\/li>\n<\/ul>\n<\/li>\n<li>income\n<ul>\n<li>Income of the voter.<\/li>\n<\/ul>\n<\/li>\n<li>vote\n<ul>\n<li>The vote is the target which we\u00a0are going to predict using the trained logistic regression model.<\/li>\n<li>vote having two possible outcomes: 0 means Clinton, 1 means Dole.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n<h3>Data creation for modeling and testing<\/h3>\n<h4>Selecting the features<\/h4>\n<p style=\"text-align: justify;\">Out of the above features (headers), we are going to use only the below headers. To select the best features from all the available features we use the feature engineering concepts.<\/p>\n<p style=\"text-align: justify;\">As the feature engineering concepts too board to explain we are going to use the below, selected features which are logically having the high chance in predicting to whom the voter will vote.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Training features and target\">training_features = ['TVnews', 'PID', 'age', 'educ', 'income']\r\ntarget = 'vote'<\/pre>\n<p>&nbsp;<\/p>\n<p>For training the logistic regression model we are going feature in the <strong>training_fearures<\/strong> and the <strong>target<\/strong>.<\/p>\n<h4>Split the data into train and test dataset<\/h4>\n<pre class=\"lang:python decode:true \" title=\"Split dataset into train and test dataset\">def main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n\r\n    training_features = ['TVnews', 'PID', 'age', 'educ', 'income']\r\n    target = 'vote'\r\n    \r\n    # Train , Test data split\r\n    train_x, test_x, train_y, test_y = train_test_split(dataset[training_features], dataset[target], train_size=0.7)\r\n    \r\n    print \"train_x size :: \", train_x.shape\r\n    print \"train_y size :: \", train_y.shape\r\n\r\n    print \"test_x size :: \", test_x.shape\r\n    print \"test_y size :: \", test_y.shape\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p>&nbsp;<\/p>\n<ul>\n<li>To split the dataset into train and test dataset we are using the scikit-learn(sk-learn) method <strong>train_test_split<\/strong> with selected training features data and the target.<\/li>\n<li>We are using the train_size as <strong>0.7<\/strong> which means out of the all the observation\u00a0considering <strong>70%<\/strong> of observation for training and remaining <strong>30%<\/strong> for testing.<\/li>\n<li>The train_test_split give four outputs which are train_x, test_x, train_y and test_y.<\/li>\n<li>For know the size of each of the about four outputs we are printing the shape.<\/li>\n<\/ul>\n<h4>Script Output:<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Script Output\">train_x size ::  (660, 5)\r\ntrain_y size ::  (660,)\r\ntest_x size ::  (284, 5)\r\ntest_y size ::  (284,)<\/pre>\n<h3>Understanding the training data<\/h3>\n<p style=\"text-align: justify;\">For understanding the training data features, Let&#8217;s look at the each possible\u00a0value for each feature data and how the relation with target classes(0 for Clinton, 1 for Dole).<\/p>\n<p style=\"text-align: justify;\">we can find the relation between the feature and the target with a histogram. To create the histogram we need the frequencies count of each possible value of the feature and with the target classes.<\/p>\n<p style=\"text-align: justify;\">Let me explain what I am talking about with an example.<\/p>\n<p style=\"text-align: justify;\"><strong>Example:<\/strong><\/p>\n<p style=\"text-align: justify;\">Suppose we have two features for building a classification model which predicts, will the student gets <strong>A grade\u00a0<\/strong>in the exam by considering the two features. Let&#8217;s say the features are the number of study classes attended in a day and gender of the student. To create the histogram to find the relation between gender and the target A grade or not, we need frequencies like the below.<\/p>\n<p style=\"text-align: justify;\">As the feature having two possible values <strong>boy or girl<\/strong> and the target also having the two possible outcomes <strong>A<\/strong> <strong>grade or not<\/strong><\/p>\n<ul>\n<li style=\"text-align: justify;\">Frequencies count of the <strong>boy<\/strong> and <strong>A grade<\/strong><\/li>\n<li style=\"text-align: justify;\">Frequencies count of the <strong>boy<\/strong> and <strong>Not<\/strong><\/li>\n<li style=\"text-align: justify;\">Frequencies count of the <strong>girl<\/strong> and <strong>A grade<\/strong><\/li>\n<li style=\"text-align: justify;\">Frequencies count of the <strong>girl<\/strong> and <strong>Not<\/strong><\/li>\n<\/ul>\n<p>So now let&#8217;s write a function with takes the dataset\u00a0feature header and target to get the about kind of frequencies results.<\/p>\n<h4>Frequencies on feature and target relation<\/h4>\n<pre class=\"lang:python decode:true \" title=\"Feature and target relationship\">def unique_observations(dataset, header, method=1):\r\n    \"\"\"\r\n    To get unique observations in the loaded pandas DataFrame column\r\n    :param dataset:\r\n    :param header:\r\n    :param method: Method to perform the unique (default method=1 for pandas and method=0 for numpy )\r\n    :return:\r\n    \"\"\"\r\n    try:\r\n        if method == 0:\r\n            # With Numpy\r\n            observations = np.unique(dataset[[header]])\r\n        elif method == 1:\r\n            # With Pandas\r\n            observations = pd.unique(dataset[header].values.ravel())\r\n        else:\r\n            observations = None\r\n            print \"Wrong method type, Use 1 for pandas and 0 for numpy\"\r\n    except Exception as e:\r\n        observations = None\r\n        print \"Error: {error_msg} \/n Please check the inputs once..!\".format(error_msg=e.message)\r\n    return observations\r\n\r\n\r\ndef feature_target_frequency_relation(dataset, f_t_headers):\r\n\r\n    \"\"\"\r\n    To get the frequency relation between targets and the unique feature observations\r\n    :param dataset:\r\n    :param f_t_headers: feature and target header\r\n    :return: feature unique observations dictionary of frequency count dictionary\r\n    \"\"\"\r\n\r\n    feature_unique_observations = unique_observations(dataset, f_t_headers[0])\r\n    unique_targets = unique_observations(dataset, f_t_headers[1])\r\n\r\n    frequencies = {}\r\n    for feature in feature_unique_observations:\r\n        frequencies[feature] = {unique_targets[0]: len(\r\n            dataset[(dataset[f_t_headers[0]] == feature) &amp; (dataset[f_t_headers[1]] == unique_targets[0])]),\r\n            unique_targets[1]: len(\r\n                dataset[(dataset[f_t_headers[0]] == feature) &amp; (dataset[f_t_headers[1]] == unique_targets[1])])}\r\n    return frequencies<\/pre>\n<p style=\"text-align: justify;\">To get the frequencies relation between target and feature, we written two functions\u00a0<strong>unique_observations<\/strong> and <strong>feature_target_frequency_relation<\/strong><\/p>\n<p style=\"text-align: justify;\"><strong>The unique_observation<\/strong> function takes the dataset and header as input parameters and returns the unique values in the dataset for that header. Suppose if the data is [1, 2, 1, 2, 3, 5, 1, 4] then output will be [1, 2, 3, 4, 5]<\/p>\n<p style=\"text-align: justify;\">Next function is the <strong>feature_target_frequency_relation\u00a0<\/strong>will take dataset and header and target header as an input parameter and returns the frequencies.<\/p>\n<p style=\"text-align: justify;\">Let&#8217;s run the fearue_target_frequency_relation function with feature header (&#8220;educ&#8221;) and target feature (&#8220;vote&#8221;)<\/p>\n<pre class=\"lang:python decode:true\" title=\"educ and vote frequencies \">def main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n\r\n    training_features = ['TVnews', 'PID', 'age', 'educ', 'income']\r\n    target = 'vote'\r\n\r\n    # Train , Test data split\r\n    train_x, test_x, train_y, test_y = train_test_split(dataset[training_features], dataset[target], train_size=0.7)\r\n\r\n    print \"edu_target_frequencies :: \", feature_target_frequency_relation(dataset, [training_features[3], target])\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p>&nbsp;<\/p>\n<h4>Script Output<\/h4>\n<pre class=\"lang:python decode:true \" title=\"Script Output\">edu_target_frequencies ::  {1: {0: 10, 1: 3}, 2: {0: 38, 1: 14}, 3: {0: 153, 1: 95}, 4: {0: 106, 1: 81}, 5: {0: 53, 1: 37}, 6: {0: 119, 1: 108}, 7: {0: 72, 1: 55}}<\/pre>\n<p style=\"text-align: justify;\">This seems interesting in the training dataset the feature education (<strong>educ<\/strong>) \u00a0value is 1 for 13 (10 + 3) time and out of 13, 10 votes for <strong>Clinton<\/strong> and only 3 votes for the <strong>dole<\/strong>. Which is the\u00a0strong signal for our classifier\u00a0while predicting to whom the voter will vote.<\/p>\n<p style=\"text-align: justify;\">Now we are ready with frequencies so we need to write a function which takes the calculated frequencies as input store the histogram.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Histogram\">def feature_target_histogram(feature_target_frequencies, feature_header):\r\n    \"\"\"\r\n    \r\n    :param feature_target_frequencies: \r\n    :param feature_header: \r\n    :return: \r\n    \"\"\"\r\n    keys = feature_target_frequencies.keys()\r\n    y0 = [feature_target_frequencies[key][0] for key in keys]\r\n    y1 = [feature_target_frequencies[key][1] for key in keys]\r\n    trace1 = go.Bar(\r\n        x=keys,\r\n        y=y0,\r\n        name='Clinton'\r\n    )\r\n    trace2 = go.Bar(\r\n        x=keys,\r\n        y=y1,\r\n        name='Dole'\r\n    )\r\n\r\n    data = [trace1, trace2]\r\n    layout = go.Layout(\r\n        barmode='group',\r\n        title='Feature :: ' + feature_header + ' Clinton Vs Dole votes Frequency',\r\n        xaxis=dict(title=\"Feature :: \" + feature_header + \" classes\"),\r\n        yaxis=dict(title=\"Votes Frequency\")\r\n    )\r\n    fig = go.Figure(data=data, layout=layout)\r\n    # plot_url = py.plot(fig, filename=feature_header + ' - Target - Histogram')\r\n    py.image.save_as(fig, filename=feature_header + '_Target_Histogram.png')<\/pre>\n<p>&nbsp;<\/p>\n<p style=\"text-align: justify;\">Don&#8217;t get scared about the code it&#8217;s just the histogram template form plotly. We just need to few modifications to the template for our needs. Once the modification did we just need to call the function with the proper inputs.<\/p>\n<p style=\"text-align: justify;\">As the target (vote) having two possible outcomes, we need to compare the relation wth histograms. For that I am getting the results in <strong>keys, y0, y1\u00a0<\/strong>for feature (educ) these are the results for keys, y0, y1<\/p>\n<pre class=\"lang:python decode:true \" title=\"Histogram inputs for edu\">edu_target_frequencies ::  {1: {0: 10, 1: 3}, 2: {0: 38, 1: 14}, 3: {0: 153, 1: 95}, 4: {0: 106, 1: 81}, 5: {0: 53, 1: 37}, 6: {0: 119, 1: 108}, 7: {0: 72, 1: 55}}\r\n\r\nkeys ::  [1, 2, 3, 4, 5, 6, 7]\r\ny0 ::  [10, 38, 153, 106, 53, 119, 72]\r\ny1 ::  [3, 14, 95, 81, 37, 108, 55]<\/pre>\n<p>If you observe the edu_target_frequencies and the keys, y0, y1 you can clearly understand what we are trying to do here.<\/p>\n<p>Now let&#8217;s call the <strong>feature_target_histogram<\/strong>\u00a0function for all the tranning_feature and check out the results.<\/p>\n<pre class=\"lang:python decode:true\" title=\"Histogram for all the training features\">def main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n\r\n    training_features = ['TVnews', 'PID', 'age', 'educ', 'income']\r\n    target = 'vote'\r\n\r\n    # Train , Test data split\r\n    train_x, test_x, train_y, test_y = train_test_split(dataset[training_features], dataset[target], train_size=0.7)\r\n\r\n    for feature in training_features:\r\n        feature_target_frequencies = feature_target_frequency_relation(dataset, [feature, target])\r\n        feature_target_histogram(feature_target_frequencies, feature)\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p>Below are the stored histogram images after running the above code.<\/p>\n<h4>TVnews and Target(vote) histogram<\/h4>\n<div id=\"attachment_2868\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2868\" class=\"size-full wp-image-2868 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/TVnews_Target_Histogram.png\" alt=\"Logistic regression TVnews Target Histogram\" width=\"700\" height=\"500\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/TVnews_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/TVnews_Target_Histogram-300x214.png 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/500;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2868\" class=\"size-full wp-image-2868\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/TVnews_Target_Histogram.png\" alt=\"Logistic regression TVnews Target Histogram\" width=\"700\" height=\"500\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/TVnews_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/TVnews_Target_Histogram-300x214.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2868\" class=\"wp-caption-text\">TVnews Target Histogram<\/p><\/div>\n<h4>PID and Target(vote) histogram<\/h4>\n<div id=\"attachment_2869\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2869\" class=\"size-full wp-image-2869 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/PID_Target_Histogram.png\" alt=\"Logistic regression PID Target Histogram\" width=\"700\" height=\"500\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/PID_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/PID_Target_Histogram-300x214.png 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/500;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2869\" class=\"size-full wp-image-2869\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/PID_Target_Histogram.png\" alt=\"Logistic regression PID Target Histogram\" width=\"700\" height=\"500\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/PID_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/PID_Target_Histogram-300x214.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2869\" class=\"wp-caption-text\">PID Target Histogram<\/p><\/div>\n<h4>Age and Target(vote) histogram<\/h4>\n<div id=\"attachment_2870\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2870\" class=\"size-full wp-image-2870 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/age_Target_Histogram.png\" alt=\"Logistic regression Age Target Histogram\" width=\"700\" height=\"500\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/age_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/age_Target_Histogram-300x214.png 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/500;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2870\" class=\"size-full wp-image-2870\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/age_Target_Histogram.png\" alt=\"Logistic regression Age Target Histogram\" width=\"700\" height=\"500\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/age_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/age_Target_Histogram-300x214.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2870\" class=\"wp-caption-text\">Age Target Histogram<\/p><\/div>\n<h4>Education and Target(vote) histogram<\/h4>\n<div id=\"attachment_2872\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2872\" class=\"size-full wp-image-2872 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/educ_Target_Histogram.png\" alt=\"Logistic regression Educ Target Histogram\" width=\"700\" height=\"500\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/educ_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/educ_Target_Histogram-300x214.png 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/500;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2872\" class=\"size-full wp-image-2872\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/educ_Target_Histogram.png\" alt=\"Logistic regression Educ Target Histogram\" width=\"700\" height=\"500\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/educ_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/educ_Target_Histogram-300x214.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2872\" class=\"wp-caption-text\">Educ Target Histogram<\/p><\/div>\n<h4>Income and Target(vote) histogram<\/h4>\n<div id=\"attachment_2873\" style=\"width: 710px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2873\" class=\"size-full wp-image-2873 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/income_Target_Histogram.png\" alt=\"Logistic regression Income Target Histogram\" width=\"700\" height=\"500\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/income_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/income_Target_Histogram-300x214.png 300w\" data-sizes=\"(max-width: 700px) 100vw, 700px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 700px; --smush-placeholder-aspect-ratio: 700\/500;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-2873\" class=\"size-full wp-image-2873\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/income_Target_Histogram.png\" alt=\"Logistic regression Income Target Histogram\" width=\"700\" height=\"500\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/income_Target_Histogram.png 700w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/income_Target_Histogram-300x214.png 300w\" sizes=\"(max-width: 700px) 100vw, 700px\" \/><\/noscript><p id=\"caption-attachment-2873\" class=\"wp-caption-text\">Income Target Histogram<\/p><\/div>\n<p style=\"text-align: justify;\">Please spend some time on understanding each histogram and how the relation with the target. Now let implement the logistic regression model in python with selected training features and the target.<\/p>\n<h3>Implementing the logistic regression model in python with scikit-learn<\/h3>\n<pre class=\"lang:python decode:true\" title=\"Traning the logistic regression model\">def train_logistic_regression(train_x, train_y):\r\n    \"\"\"\r\n    Training logistic regression model with train dataset features(train_x) and target(train_y)\r\n    :param train_x:\r\n    :param train_y:\r\n    :return:\r\n    \"\"\"\r\n\r\n    logistic_regression_model = LogisticRegression()\r\n    logistic_regression_model.fit(train_x, train_y)\r\n    return logistic_regression_model\r\n\r\n\r\ndef main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n\r\n    training_features = ['TVnews', 'PID', 'age', 'educ', 'income']\r\n    target = 'vote'\r\n\r\n    # Train , Test data split\r\n    train_x, test_x, train_y, test_y = train_test_split(dataset[training_features], dataset[target], train_size=0.7)\r\n\r\n    # Training Logistic regression model\r\n    trained_logistic_regression_model = train_logistic_regression(train_x, train_y)\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p style=\"text-align: justify;\">To implement the logistic regression model we created the function train_logistic_regression with train_x and train_y as input parameters. With this logistic regression model created and trained with the training dataset. Now let&#8217;s chek out the accuracies of the model.<\/p>\n<h3>Logistic regression model \u00a0accuracy calculation<\/h3>\n<p style=\"text-align: justify;\">Let&#8217;s write the function which takes the trained logistic regression model\u00a0<strong>feature values<\/strong> (train_x or test_x) and <strong>target values<\/strong> (train_y or test_y ) for calculating the accuracy.<\/p>\n<pre class=\"lang:python decode:true \" title=\"Function for calculating the accuracies\">def model_accuracy(trained_model, features, targets):\r\n    \"\"\"\r\n    Get the accuracy score of the model\r\n    :param trained_model:\r\n    :param features:\r\n    :param targets:\r\n    :return:\r\n    \"\"\"\r\n    accuracy_score = trained_model.score(features, targets)\r\n    return accuracy_score<\/pre>\n<p style=\"text-align: justify;\">This function will take the trained model, features and targets as input. Uses the trained_modle and the features to predict the targets and the compare with the actual targets and returns the accuracy score.<\/p>\n<p style=\"text-align: justify;\">Now let&#8217;s \u00a0call the above function with train_x \u00a0and train_y for getting accuracies of our model on train dataset and later call the same function with test_x and test_y for getting accuracies of our model the on test dataset.<\/p>\n<h4 style=\"text-align: justify;\">Logistic regression model accuracy on train dataset<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Logistic Regression model accuracy\">train_accuracy = model_accuracy(trained_logistic_regression_model, train_x, train_y)\r\n\r\nprint \"Train Accuracy :: \", train_accuracy<\/pre>\n<h4>Script output:<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Train Accuracy\">Train Accuracy ::  0.901515151515<\/pre>\n<h4 style=\"text-align: justify;\">Logistic regression model accuracy on test dataset<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Logistic regression model test accuracy\"># Testing the logistic regression model\r\ntest_accuracy = model_accuracy(trained_logistic_regression_model, test_x, test_y)\r\n\r\nprint \"Test Accuracy :: \", test_accuracy<\/pre>\n<h4>Script output:<\/h4>\n<pre class=\"lang:python decode:true\" title=\"Train Accuracy\">Test Accuracy ::  0.911971830986<\/pre>\n<p style=\"text-align: justify;\">With the selected training features we got a test accuracy <strong>91%<\/strong>. Play will different features and let me know the test accuracy you got in the comments.<\/p>\n<p style=\"text-align: justify;\">We can save this trained logistic regression model to use in some other applications without importing the major libraries. Check out <a href=\"https:\/\/dataaspirant.com\/2017\/02\/13\/save-scikit-learn-models-with-python-pickle\/\" target=\"_blank\">how to dump and load the trained classifier article<\/a>.<\/p>\n<p style=\"text-align: justify;\">Use the scikit learn <strong>predict<\/strong> method to predict, whom the voter will vote. Given the voter features and let me know the results in the comments section. If you face any difficulty in using the predict method, Do check out how I use predict method in <a href=\"https:\/\/dataaspirant.com\/2017\/02\/01\/decision-tree-algorithm-python-with-scikit-learn\/\" target=\"_blank\">implementing decision tree classifier in python<\/a>.<\/p>\n<h3 style=\"text-align: justify;\">Logistic regression model complete code<\/h3>\n<pre class=\"lang:python decode:true \" title=\"Logistic regression model complete code\">#!\/usr\/bin\/env python\r\n# logistic_regression.py\r\n# Author : Saimadhu\r\n# Date: 19-March-2017\r\n# About: Implementing Logistic Regression Classifier to predict to whom the voter will vote.\r\n\r\n# Required Python Packages\r\nimport pandas as pd\r\nimport numpy as np\r\nimport pdb\r\nimport plotly.plotly as py\r\nimport plotly.graph_objs as go\r\n\r\n# import plotly.plotly as py\r\n# from plotly.graph_objs import *\r\npy.sign_in('dataaspirant', 'RhJdlA1OsXsTjcRA0Kka')\r\n\r\nfrom sklearn.cross_validation import train_test_split\r\nfrom sklearn.linear_model import LogisticRegression\r\nfrom sklearn import metrics\r\n\r\n\r\n# Files\r\nDATA_SET_PATH = \"..\/Inputs\/anes_dataset.csv\"\r\n\r\n\r\ndef dataset_headers(dataset):\r\n    \"\"\"\r\n    To get the dataset header names\r\n    :param dataset: loaded dataset into pandas DataFrame\r\n    :return: list of header names\r\n    \"\"\"\r\n    return list(dataset.columns.values)\r\n\r\n\r\ndef unique_observations(dataset, header, method=1):\r\n    \"\"\"\r\n    To get unique observations in the loaded pandas DataFrame column\r\n    :param dataset:\r\n    :param header:\r\n    :param method: Method to perform the unique (default method=1 for pandas and method=0 for numpy )\r\n    :return:\r\n    \"\"\"\r\n    try:\r\n        if method == 0:\r\n            # With Numpy\r\n            observations = np.unique(dataset[[header]])\r\n        elif method == 1:\r\n            # With Pandas\r\n            observations = pd.unique(dataset[header].values.ravel())\r\n        else:\r\n            observations = None\r\n            print \"Wrong method type, Use 1 for pandas and 0 for numpy\"\r\n    except Exception as e:\r\n        observations = None\r\n        print \"Error: {error_msg} \/n Please check the inputs once..!\".format(error_msg=e.message)\r\n    return observations\r\n\r\n\r\ndef feature_target_frequency_relation(dataset, f_t_headers):\r\n\r\n    \"\"\"\r\n    To get the frequency relation between targets and the unique feature observations\r\n    :param dataset:\r\n    :param f_t_headers: feature and target header\r\n    :return: feature unique observations dictionary of frequency count dictionary\r\n    \"\"\"\r\n\r\n    feature_unique_observations = unique_observations(dataset, f_t_headers[0])\r\n    unique_targets = unique_observations(dataset, f_t_headers[1])\r\n\r\n    frequencies = {}\r\n    for feature in feature_unique_observations:\r\n        frequencies[feature] = {unique_targets[0]: len(\r\n            dataset[(dataset[f_t_headers[0]] == feature) &amp; (dataset[f_t_headers[1]] == unique_targets[0])]),\r\n            unique_targets[1]: len(\r\n                dataset[(dataset[f_t_headers[0]] == feature) &amp; (dataset[f_t_headers[1]] == unique_targets[1])])}\r\n    return frequencies\r\n\r\n\r\ndef feature_target_histogram(feature_target_frequencies, feature_header):\r\n    \"\"\"\r\n\r\n    :param feature_target_frequencies:\r\n    :param feature_header:\r\n    :return:\r\n    \"\"\"\r\n    keys = feature_target_frequencies.keys()\r\n    y0 = [feature_target_frequencies[key][0] for key in keys]\r\n    y1 = [feature_target_frequencies[key][1] for key in keys]\r\n\r\n    trace1 = go.Bar(\r\n        x=keys,\r\n        y=y0,\r\n        name='Clinton'\r\n    )\r\n    trace2 = go.Bar(\r\n        x=keys,\r\n        y=y1,\r\n        name='Dole'\r\n    )\r\n    data = [trace1, trace2]\r\n    layout = go.Layout(\r\n        barmode='group',\r\n        title='Feature :: ' + feature_header + ' Clinton Vs Dole votes Frequency',\r\n        xaxis=dict(title=\"Feature :: \" + feature_header + \" classes\"),\r\n        yaxis=dict(title=\"Votes Frequency\")\r\n    )\r\n    fig = go.Figure(data=data, layout=layout)\r\n    # plot_url = py.plot(fig, filename=feature_header + ' - Target - Histogram')\r\n    py.image.save_as(fig, filename=feature_header + '_Target_Histogram.png')\r\n\r\n\r\ndef train_logistic_regression(train_x, train_y):\r\n    \"\"\"\r\n    Training logistic regression model with train dataset features(train_x) and target(train_y)\r\n    :param train_x:\r\n    :param train_y:\r\n    :return:\r\n    \"\"\"\r\n\r\n    logistic_regression_model = LogisticRegression()\r\n    logistic_regression_model.fit(train_x, train_y)\r\n    return logistic_regression_model\r\n\r\n\r\ndef model_accuracy(trained_model, features, targets):\r\n    \"\"\"\r\n    Get the accuracy score of the model\r\n    :param trained_model:\r\n    :param features:\r\n    :param targets:\r\n    :return:\r\n    \"\"\"\r\n    accuracy_score = trained_model.score(features, targets)\r\n    return accuracy_score\r\n\r\n\r\ndef main():\r\n    \"\"\"\r\n    Logistic Regression classifier main\r\n    :return:\r\n    \"\"\"\r\n    # Load the data set for training and testing the logistic regression classifier\r\n    dataset = pd.read_csv(DATA_SET_PATH)\r\n    print \"Number of Observations :: \", len(dataset)\r\n\r\n    # Get the first observation\r\n    print dataset.head()\r\n\r\n    headers = dataset_headers(dataset)\r\n    print \"Data set headers :: {headers}\".format(headers=headers)\r\n\r\n    training_features = ['TVnews', 'PID', 'age', 'educ', 'income']\r\n    target = 'vote'\r\n\r\n    # Train , Test data split\r\n    train_x, test_x, train_y, test_y = train_test_split(dataset[training_features], dataset[target], train_size=0.7)\r\n    print \"train_x size :: \", train_x.shape\r\n    print \"train_y size :: \", train_y.shape\r\n\r\n    print \"test_x size :: \", test_x.shape\r\n    print \"test_y size :: \", test_y.shape\r\n\r\n    print \"edu_target_frequencies :: \", feature_target_frequency_relation(dataset, [training_features[3], target])\r\n\r\n    for feature in training_features:\r\n        feature_target_frequencies = feature_target_frequency_relation(dataset, [feature, target])\r\n        feature_target_histogram(feature_target_frequencies, feature)\r\n\r\n    # Training Logistic regression model\r\n    trained_logistic_regression_model = train_logistic_regression(train_x, train_y)\r\n    \r\n    train_accuracy = model_accuracy(trained_logistic_regression_model, train_x, train_y)\r\n\r\n    # Testing the logistic regression model\r\n    test_accuracy = model_accuracy(trained_logistic_regression_model, test_x, test_y)\r\n\r\n    print \"Train Accuracy :: \", train_accuracy\r\n    print \"Test Accuracy :: \", test_accuracy\r\n\r\nif __name__ == \"__main__\":\r\n    main()<\/pre>\n<p>You can get the complete code in <a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/tree\/master\/Logistic_Regression\/Logistic_Binary_Classification\" target=\"_blank\">Dataaspirant Github<\/a><\/p>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>|\u00a0<span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span>\u00a0|\u00a0<span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span>\u00a0|\u00a0<span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0|\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions, then feel free to comment below.\u00a0 If you want me to write on one particular topic, then do tell it to me in the comments below.<\/p>\n<h4>Related Data Science Courses<\/h4>\n<ul>\n<li><a class=\"thirstylink\" title=\"Data-Science-and-Machine-Learning-with-Python-Hands-On-Course-Udemy\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/data-science-machine-learning-python-hands-course-udemy\/\" rel=\"nofollow\" target=\"_blank\">Data Science and Machine Learning with Python<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"course-2\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-z-hands-python-r-data-science-course-udemy\/\" rel=\"nofollow\" target=\"_blank\">Complete Machine learning course with Python and R<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"course-3\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-data-analysis-coursera\/\" rel=\"nofollow\" target=\"_blank\">Machine learning and Classification models<\/a><\/li>\n<li><a class=\"thirstylink\" title=\"course-4\" href=\"http:\/\/dataaspirant.com\/recommends\/data-science-courses\/machine-learning-classification-algorithms-case-study-coursera\/\" rel=\"nofollow\" target=\"_blank\">Classification Algorithms Case study<\/a><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the last few articles, we talked about different classification algorithms. For every classification algorithm, we learn the background concepts of the algorithm and in the followed article we used the background concepts of the algorithm to build the classification model. Later we used the mode to perform the\u00a0regression or classification tasks. Likewise in this<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/implement-logistic-regression-model-python-binary-classification\/\">+ Read More<\/a><\/p>\n","protected":false},"author":2,"featured_media":2801,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[6,8],"tags":[39,1016],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/04\/Predicting-Election-Results-with-Logistic-Regression-model.jpg","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/2790"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=2790"}],"version-history":[{"count":0,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/2790\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media\/2801"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=2790"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=2790"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=2790"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}