{"id":6113,"date":"2020-09-23T03:22:43","date_gmt":"2020-09-23T03:22:43","guid":{"rendered":"https:\/\/dataaspirant.com\/?p=6113"},"modified":"2023-04-04T09:59:12","modified_gmt":"2023-04-04T04:29:12","slug":"credit-card-fraud-detection-classification-algorithms-python","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/credit-card-fraud-detection-classification-algorithms-python\/","title":{"rendered":"Credit Card Fraud Detection With Classification Algorithms In Python"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><p><span><img decoding=\"async\" alt=\"Credit card fraud detection with classification algorithms\" data-id=\"6119\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"Credit card fraud detection with classification algorithms\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/1-Credit-card-fraud-detection-with-classification-algorithms.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 613px; --smush-placeholder-aspect-ratio: 613\/368;\"><noscript><img decoding=\"async\" alt=\"Credit card fraud detection with classification algorithms\" data-id=\"6119\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"Credit card fraud detection with classification algorithms\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/1-Credit-card-fraud-detection-with-classification-algorithms.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<h1>Credit Card Fraud Detection With Classification Algorithms In Python<\/h1>\n<p dir=\"ltr\" style=\"text-align: justify;\">Fraud transactions or fraudulent activities are significant issues in many industries like <strong>banking, insurance<\/strong>, etc. Especially for the banking industry, credit card fraud detection is a pressing issue to resolve.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">These industries suffer too much due to fraudulent activities towards <strong>revenue growth and lose customer\u2019s trust<\/strong>. So these companies need to find fraud transactions before it becomes a big problem for them. &nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Unlike the other machine learning problems, in credit card fraud detection the target class distribution is <strong>not equally<\/strong> distributed. It is popularly known as the <strong>class imbalance problem<\/strong> or unbalanced data issue.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This makes this problem even more challenging to solve.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">So In this article, we will explain to you how to build credit card fraud detection using different <strong><a href=\"https:\/\/dataaspirant.com\/classification-clustering-alogrithms\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">machine learning classification algorithms<\/a>. <\/strong><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Such as,<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\"><strong><a href=\"https:\/\/dataaspirant.com\/how-decision-tree-algorithm-works\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Decision trees algorithm<\/a><\/strong><\/li>\n<li dir=\"ltr\"><strong><a href=\"https:\/\/dataaspirant.com\/random-forest-algorithm-machine-learing\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Random forest algorithm<\/a><\/strong><\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">You will also get an idea about the impact of unbalanced data on the model\u2019s performance.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let us give you the list of contents that we will discuss in the next few minutes. Just to give you a glimpse about the topics that you are going to learn from this article.<\/p>\n<p style=\"text-align: justify;\">Let\u2019s begin the discussion by understanding why we need to find fraudulent transactions\/activities in any industry.<\/p>\n<h2 id=\"t-1600792769584\">Why do we need to find fraud transactions?<\/h2>\n<p><span><img decoding=\"async\" alt=\"Credit Card Fraudulent Transactions Percentanges\" data-id=\"6127\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"Credit Card Fraudulent Transactions Percentanges\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/2-Credit-Card-Fraudulent-Transactions-Percentanges.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 613px; --smush-placeholder-aspect-ratio: 613\/368;\"><noscript><img decoding=\"async\" alt=\"Credit Card Fraudulent Transactions Percentanges\" data-id=\"6127\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"Credit Card Fraudulent Transactions Percentanges\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/2-Credit-Card-Fraudulent-Transactions-Percentanges.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Credit Card Fraudulent Transactions Percentanges<\/p>\n<p>For many companies, fraud detection is a big problem because they find these fraudulent activities after they experience <strong>high loss<\/strong>.&nbsp;<\/p>\n<p>Fraud activities happen in all&nbsp; industries. We can&#8217;t say only particular companies\/industries suffer from these fraudulent activities or transactions.&nbsp;<\/p>\n<p>But when it comes to <strong>financial-related<\/strong> companies, this fraud transaction becomes more of an issue\/problem. &nbsp;So these companies want to detect fraud transactions before the fraud activities turn into significant damage to their company.<\/p>\n<p>In the current generation, with high-end technology, still, on every 100 credit card transactions, <strong>13% are falling<\/strong> into the fraudulent activities reported by the creditcards website.<\/p>\n<p>A survey paper mentioned that in the <strong>year 1997, 63%<\/strong> of companies experienced one fraud in the past two years, and in another <strong>year 1999, <\/strong><strong>57%<\/strong> of companies experienced at least one fraud in the last one year.&nbsp;<\/p>\n<p>Here the point is not only fraud activities increase, but the way of doing scams also increases badly.&nbsp;<\/p>\n<p>Companies suffer from detecting fraud, and due to these fraudulent activities, many companies worldwide have lost billions of dollars yearly.<\/p>\n<p>And one more thing, for any company, customer&#8217;s trust is more important to achieve or reach some position in the business marketplace. If a company cannot find these fraudulent activities, companies lose customer&#8217;s trust; then, they will suffer from customer churn.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600792769585\">Fraud Detection Approaches<\/h3>\n<p>So companies start to detect these fraud activities automatically by using smart technologies.&nbsp;<\/p>\n<p>First, companies <strong>hire few people<\/strong> only for the detection of these kinds of activities or transactions. But here they must and should be experts in this field or domain, and also the team should have knowledge of how frauds occur in particular domains. This requires more resources, such as people&#8217;s effort and time.<\/p>\n<p>Second, companies <strong>changed manual<\/strong> processes to rule-based solutions. But this one also fails most of the time to detect frauds.&nbsp;<\/p>\n<p>Because in the real world, the way of doing frauds is changing drastically day by day. These rule-based systems follow some <strong>rules and conditions<\/strong>. If a new fraud process is different from others, then these systems fail. It requires adding that new rule to code and execute.&nbsp;<\/p>\n<p>Now companies are trying to adopt Artificial Intelligence or <a href=\"https:\/\/dataaspirant.com\/supervised-learning-algorithms\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>machine learning algorithms<\/strong><\/a> to detect frauds. Machine learning algorithms performed very well for this type of problem. &nbsp;<\/p>\n<p style=\"text-align: justify;\">The payment gateway Stripe, for example \u2014 which can be integrated with the recurring payment provider <a href=\"https:\/\/www.chargebee.com\/recurring-payments\/\" style=\"outline: none;\" target=\"_blank\" rel=\"noopener\">Chargebee<\/a> \u2014 uses an adaptive machine learning algorithm that evaluates risk in real-time and predicts whether a payment is likely to be fraudulent.&nbsp;<\/p>\n<h2 dir=\"ltr\" id=\"t-1600792769586\">What is Credit Card Fraud Detection?<\/h2>\n<p><span><img decoding=\"async\" alt=\"Credit Card Fraud Detection\" data-id=\"6130\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit Card Fraud Detection\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/3-Credit-Card-Fraud-Detection.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;\"><noscript><img decoding=\"async\" alt=\"Credit Card Fraud Detection\" data-id=\"6130\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit Card Fraud Detection\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/3-Credit-Card-Fraud-Detection.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Credit Card Fraud Detection<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the above section, we discussed the need for identifying fraudulent activities. The credit card fraud classification problem is used to find fraud transactions or fraudulent activities before they become a major problem to credit card companies.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">It uses the combination of <strong>fraud and non-fraud transactions<\/strong> from the historical data with different people&#8217;s credit card transaction data to estimate fraud or non-fraud on credit card transactions.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In this article, we are using the <strong>popular credit card dataset<\/strong>. Let\u2019s understand the data before we start building the fraud detection models.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600792769587\">Understanding of Credit Card Dataset&nbsp;<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">For this credit card fraud classification problem, we are using the dataset which was downloaded from the <strong>Kaggle platform.<\/strong>&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">You can find and download the dataset from <a href=\"https:\/\/www.kaggle.com\/mlg-ulb\/creditcardfraud\" target=\"_blank\">here<\/a>.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Before going to the model development part, we should have some knowledge about our dataset.<\/p>\n<p><span><img decoding=\"async\" alt=\"Understand Credit Card Dataset\" data-id=\"6133\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Understand Credit Card Dataset\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/4-Understand-Credit-Card-Dataset.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;\"><noscript><img decoding=\"async\" alt=\"Understand Credit Card Dataset\" data-id=\"6133\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Understand Credit Card Dataset\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/4-Understand-Credit-Card-Dataset.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Understand Credit Card Dataset<\/p>\n<p dir=\"ltr\">Such as&nbsp;<\/p>\n<p ltr\"\"=\"\">\n<ol>\n<li dir=\"ltr\">What is the size of the dataset?<\/li>\n<li dir=\"ltr\">How many features does the dataset have?<\/li>\n<li dir=\"ltr\">What are the target values?<\/li>\n<li dir=\"ltr\">How many samples under each target value? , etc.<\/li>\n<\/ol>\n<p dir=\"ltr\" style=\"text-align: justify;\">If we know some information about the dataset, then we can decide what we have to do?.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">What are the questions we discussed above, all &nbsp;we can explore by using the python <strong>pandas library<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let&#8217;s jump to the data exploration part to find answers to all questions we have.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600792769588\">Data Explorations<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">First, we need to load the dataset. After downloading the dataset, extract the data and keep the file in the dataset under the project folder.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can quickly load it using pandas.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Our dataset is a CSV(Comma Separated Values) file. We can use the <strong>read_csv<\/strong> function from pandas to read the file.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Ok, now find the answers for our above dataset related questions.<\/p>\n<\/p>\n<p style=\"text-align: justify;\">Dataset has <strong>284807 rows and 31 features<\/strong>. The result of the shape variable is a tuple that has the number of rows, number of columns of the dataset.<\/p>\n<p style=\"text-align: justify;\">We can see how the dataset looks like. The below command showcases &nbsp;only five rows, <strong>head()<\/strong> by default, gives <strong>5<\/strong> samples.&nbsp;<\/p>\n<p><span><img decoding=\"async\" alt=\"credit card data observations\" data-id=\"6140\" width=\"626\" data-init-width=\"1358\" height=\"142\" data-init-height=\"308\" title=\"credit card data observations\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/credit-card-data-observations.png\" data-width=\"626\" data-height=\"142\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/142;\"><noscript><img decoding=\"async\" alt=\"credit card data observations\" data-id=\"6140\" width=\"626\" data-init-width=\"1358\" height=\"142\" data-init-height=\"308\" title=\"credit card data observations\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/credit-card-data-observations.png\" data-width=\"626\" data-height=\"142\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Credit card data observations<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If you want to see more samples from the top, pass the number representing the number of samples you want to see like <strong>fraud_df.head(10).<\/strong>&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">You can also see bottom samples by using the <strong>tail()<\/strong> function. Both are working in the same way.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can get all the list of feature names.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">From this, we know Class is the target variable, and the remaining all are features of our dataset.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let&#8217;s see what are the unique values we are having for the target variable.<\/p>\n<\/p>\n<p dir=\"ltr\">The target variable Class has <strong>0 and 1<\/strong> values. Here<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">0 for <strong>non-fraudulent<\/strong> transactions<\/li>\n<li dir=\"ltr\">1 for <strong>fraudulent<\/strong> transactions<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Because we aim to find fraudulent transactions, the dataset&#8217;s target value has a positive value for that.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Still, What is pending in data exploration questions?&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">yeah, we have to check how many samples each target class is having.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Yeah, we have <strong>284315<\/strong> non-fraudulent transaction samples &amp; <strong>492<\/strong> fraudulent transaction samples.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We will discuss more about the data in the later sections of this article.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">You are going to know the variation of this number of samples and how much impact on the <strong>model&#8217;s performance<\/strong>, how we can evaluate model performance for this data, etc.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Still, now you only know about the dataset, such<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Dataset size<\/li>\n<li dir=\"ltr\">Number of samples(rows) and features(columns)<\/li>\n<li dir=\"ltr\">Names of the features<\/li>\n<li dir=\"ltr\">About target variables, etc.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now we will discuss different data preprocessing techniques for our dataset.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The data preprocessing techniques will be completely different from the text preprocessing techniques we discussed in the <strong><a href=\"https:\/\/dataaspirant.com\/nlp-text-preprocessing-techniques-implementation-python\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">natural language processing data preprocessing techniques<\/a><\/strong> article&nbsp;<\/p>\n<h2 dir=\"ltr\" id=\"t-1600793624237\">Credit Card Data Preprocessing<\/h2>\n<p><span><img decoding=\"async\" alt=\"Credit card data preprocessing\" data-id=\"6147\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit card data preprocessing\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/5-credit-card-data-preprocessing.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;\"><noscript><img decoding=\"async\" alt=\"Credit card data preprocessing\" data-id=\"6147\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit card data preprocessing\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/5-credit-card-data-preprocessing.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Credit card data preprocessing<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Preprocessing is the process of cleaning the dataset. In this step, we will apply different methods to clean the raw data to feed more meaningful data for the modeling phase. This method includes<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Remove duplicates or irrelevant samples<\/li>\n<li dir=\"ltr\">Update missing values with the most relevant values&nbsp;<\/li>\n<li dir=\"ltr\">Convert one data type to another example, categorical to integers, etc.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Okay, now we will spend a couple of minutes checking the dataset and applying corresponding techniques to clean data.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This step aims to improve the quality of the data.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600793624238\">Removing irrelevant columns\/features<\/h3>\n<p>In our dataset, only one irrelevant or not useful feature id Time. So we can drop that feature from the dataset.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If you want to drop more features from data, call <strong>drop()<\/strong> method with a list of feature names.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can observe no feature name Time in the list of feature names after dropping the Time feature\/column.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600793624239\">Checking null or nan values&nbsp;<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can check the datatypes of all features and, at the same time, the number of non-null values of all features by using <strong>info() <\/strong>of pandas.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Null or nan values are nothing, but there is no value for that particular feature or attribute.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For example, these nan or null values are coming if the customer or <strong>user does not <\/strong>fill all information in the forms. Blank values are treated as null or nan values.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">It&#8217;s okay; we can know all this information just by using info() from pandas.<\/p>\n<\/p>\n<p dir=\"ltr\">See the result of dataset info();&nbsp;<\/p>\n<p dir=\"ltr\">it provides all information about our dataset, such as&nbsp;<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Total number of samples or rows<\/li>\n<li dir=\"ltr\">Column names<\/li>\n<li dir=\"ltr\">Number of non-null values<\/li>\n<li dir=\"ltr\">The data type of each column<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Our dataset doesn\u2019t have any null values because the total number features are 284807 that ranges from 0-284806; all features have the same number of samples\/rows.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600793624240\">Data Transformation<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">Except for the Amount column, all column\u2019s values are within some range of values. So let&#8217;s change the Amount columns values to a smaller range of numbers.&nbsp;<\/p>\n<p>We can simply do this process by using <strong>StandardScaler<\/strong> from the sklearn library.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">See the values of the <strong>Amount<\/strong> feature values are in high range compared to other feature values.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We will change values within a smaller range.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The scalar result is added as a new column with <strong>norm_amount<\/strong> name to the data frame after we drop the Amount column because there is no use with it.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600793624241\">Splitting dataset&nbsp;<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now we will take all independent columns (target column is dependent and the remaining all are independent columns to each other), as <strong>X<\/strong> and the <strong>target<\/strong> variable as <strong>y<\/strong>.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now we need to split the whole dataset into train and test dataset. Training data is used at the time of building the model and a test dataset is used to evaluate trained models.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">By using the <strong>train_test_split<\/strong> method from the sklearn library we can do this process of splitting the dataset to train and test sets.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now our dataset is ready for building models. Let&#8217;s jump to the development of &nbsp;the model using machine learning algorithms such as decision tree and <strong><a href=\"https:\/\/dataaspirant.com\/random-forest-classifier-python-scikit-learn\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">random forest classification algorithms<\/a><\/strong> from the sklearn module.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600793624242\">Building Credit Card Fraud Detection using Machine Learning algorithms<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now we can build models using different machine learning algorithms. Before creating a model, we need to find the type of problem statement, which means is supervised or unsupervised algorithms.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Our problem statement falls under the supervised learning problem means the dataset has a target value for each row or sample in the dataset.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\"><strong><a href=\"https:\/\/dataaspirant.com\/supervised-and-unsupervised-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Supervised machine learning algorithms<\/a> <\/strong>are two types&nbsp;<\/p>\n<ul>\n<li dir=\"ltr\">Classification Algorithms<\/li>\n<li dir=\"ltr\">Regression Algorithms<\/li>\n<\/ul>\n<p tve-droppable\"=\"\" dir=\"ltr\" \"=\"\" style=\"text-align: justify;\">Our problem statement belongs to what type of algorithms?&nbsp;<\/p>\n<p \"=\"\" dir=\"ltr\" style=\"text-align: justify;\" tve-droppable\"=\"\">Yeah, exactly.<\/p>\n<p \"=\"\" dir=\"ltr\" style=\"text-align: justify;\" tve-droppable\"=\"\">Credit card fraud detection is a <strong>classification<\/strong> problem. Target variable values of Classification problems have integer(0,1) or categorical values(fraud, non-fraud). The target variable of our dataset \u2018Class\u2019 has only two labels &#8211; 0 (non-fraudulent) and 1 (fraudulent).<\/p>\n<p tve-droppable\"=\"\" dir=\"ltr\" \"=\"\" style=\"text-align: justify;\">Before going further let us give an introduction for both decision tree classification and random forest classification. As in this article, we are going to use these two algorithms to build the credit card fraudulent activities identification model.<\/p>\n<ul>\n<li ltr\"\"=\"\">Decision Tree Classification Algorithm<\/li>\n<li ltr\"\"=\"\">Random Forest Classification Algorithm<\/li>\n<\/ul>\n<h3 dir=\"ltr\" id=\"t-1600793624243\">Decision Tree Algorithm Overview<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">The decision tree is the simplest and most popular classification algorithm. For building the model the decision tree algorithm considers all the provided features of the data and comes up with the <strong>important<\/strong> features.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Because of this advantage, the decision tree algorithms also used in identifying the importance of the feature metrics. Which used in <strong>handpicking<\/strong> the features.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Once the important features identified then the model trains with the training data to come up with a <strong>set of rules<\/strong>. These rules used in predicting future cases or for the test dataset.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This is a quick overview of the decision tree algorithm. If you want to learn more about the algorithm and implement in python, have a look at the below articles written by our team.<\/p>\n<ul>\n<li ltr\"\"=\"\"><strong><a href=\"https:\/\/dataaspirant.com\/how-decision-tree-algorithm-works\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">How the decision tree learns from the training data<\/a><\/strong><\/li>\n<li ltr\"\"=\"\"><strong><a href=\"https:\/\/dataaspirant.com\/decision-tree-algorithm-python-with-scikit-learn\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Decision tree algorithm implementation in python<\/a><\/strong><\/li>\n<li ltr\"\"=\"\"><strong><a href=\"https:\/\/dataaspirant.com\/decision-tree-classifier-implementation-in-r\/\" target=\"_blank\" rel=\"noopener\">Implementing decision tree in R<\/a><\/strong><\/li>\n<li ltr\"\"=\"\"><strong><a href=\"https:\/\/dataaspirant.com\/visualize-decision-tree-python-graphviz\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">How to visuvalizing the decsion tree<\/a><\/strong><\/li>\n<\/ul>\n<p dir=\"ltr\">Now let\u2019s see a quick overview of the random forest algorithm.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600793624244\">Random Forest Algorithm Overview<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">The random forest algorithm falls under the <strong>ensemble learning algorithm<\/strong> category. In the random forest algorithm, we build N decision tree models. &nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">All the models predict the target value. Using the <strong>majority voting<\/strong> approach the final target value will be predicted.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For building the individual decision tree, the random forest algorithm randomly creates the sample dataset. These sample datasets are called as the <strong>bootstrap samples<\/strong>.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Suppose we want to build the N decision trees to create the forest, the algorithm first creates N bootstrap samples. Later for each bootstrap sample, one decision tree model will build.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This is a quick overview of the random forest algorithm, If you want to learn more, please have a look at the below articles.<\/p>\n<ul>\n<li ltr\"\"=\"\"><a href=\"https:\/\/dataaspirant.com\/ensemble-methods-bagging-vs-boosting-difference\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>How bootstrap samples created in<\/strong><strong>&nbsp;Ensembleme learning methods.<\/strong><\/a><\/li>\n<li ltr\"\"=\"\"><strong><a href=\"https:\/\/dataaspirant.com\/random-forest-algorithm-machine-learing\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">End to end the working nature of the Random forest algorithm.<\/a><\/strong><\/li>\n<li ltr\"\"=\"\"><strong><a href=\"https:\/\/dataaspirant.com\/random-forest-classifier-python-scikit-learn\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Implementing the Random forest algorithm in python.<\/a><\/strong><\/li>\n<\/ul>\n<p dir=\"ltr\">Now let\u2019s go to the implementation part, the crazy one \ud83d\ude42<\/p>\n<h2 dir=\"ltr\" id=\"t-1600793624245\">Credit Card Fraud Detection with Decision Tree Algorithm<\/h2>\n<p><span><img decoding=\"async\" alt=\"Credit card fraud detection with decision tree\" data-id=\"6161\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit card fraud detection with decision tree\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/6-credit-card-fraud-detection-with-decision-tree.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;\"><noscript><img decoding=\"async\" alt=\"Credit card fraud detection with decision tree\" data-id=\"6161\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit card fraud detection with decision tree\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/6-credit-card-fraud-detection-with-decision-tree.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Credit card fraud detection with decision tree<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We will use the <strong>DecisionTreeClassifier<\/strong> class from the sklearn library to train and evaluate models. We use <strong>X_train and y_train<\/strong> data for training purposes. X_train is a training dataset with features, and y_train is the target label.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600793624246\">Decision tree algorithm Implementation using python sklearn library<\/h3>\n<\/p>\n<p>The output for the above code listed below.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Wow, our decision tree classification gives <strong>99%<\/strong> accuracy on test data.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">But why <strong>f1-score<\/strong> on label 1 too less ?.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Remember this point; we will discuss these <strong>metrics performances<\/strong> in the coming section of this article where we address the question<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Why the accuracy evaluation metric is not suitable for this problem?<\/p>\n<h2 dir=\"ltr\" id=\"t-1600793624247\">Credit Card Fraud Detection with Random Forest Algorithm<\/h2>\n<p><span><img decoding=\"async\" alt=\"Credit card fraud detection with random forest\" data-id=\"6166\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit card fraud detection with random forest\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/7-credit-card-fraud-detection-with-random-forest.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;\"><noscript><img decoding=\"async\" alt=\"Credit card fraud detection with random forest\" data-id=\"6166\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Credit card fraud detection with random forest\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/7-credit-card-fraud-detection-with-random-forest.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Credit card fraud detection with random forest<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Same as the above decision tree implementation, we use X_train and y_train dataset for training purposes and X_test for evaluation. Here we train the ensemble technique model of <strong>RandomForestClassifier<\/strong> from the sklearn. We can see the variations in the evaluation results.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600823366412\">Random forest algorithm Implementation using sklearn library<\/h3>\n<\/p>\n<p>The output for the above code listed below.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Wow, this model&#8217;s accuracy is also <strong>99%<\/strong> great, but what about remaining evaluation metrics such as precision, recall, F1-score.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let&#8217;s discuss these variations why it happens, all these in the coming section.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600823366413\">Why Accuracy not suitable for Data Imbalance Problems?<\/h2>\n<p><span><img decoding=\"async\" alt=\"How to measure performance for data imbalance problems\" data-id=\"6170\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"How to measure performance for data imbalance problems\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/8-How-to-measure-performance-for-data-imbalance-problems.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 613px; --smush-placeholder-aspect-ratio: 613\/368;\"><noscript><img decoding=\"async\" alt=\"How to measure performance for data imbalance problems\" data-id=\"6170\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"How to measure performance for data imbalance problems\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/8-How-to-measure-performance-for-data-imbalance-problems.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">How to measure performance for data imbalance problems<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">What was the reason for not applying or not considering accuracy as a performance metric for this specific problem?<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Just take some time, think about it.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Model training is completed; we got accuracy on the test set as 99%.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">But why this section?&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We are having <strong><a href=\"https:\/\/dataaspirant.com\/six-popular-classification-evaluation-metrics-in-machine-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">various classification evaluation metrics<\/a> <\/strong>to quantify the performance of the build model, accuracy is one method in that. What other methods we can apply?<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now we will discuss our dataset and what are the <strong>best evaluation metrics<\/strong> for these kinds of problems.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For this discussion, we have to remember two things that are previously discussed.<\/p>\n<p ltr\"\"=\"\">\n<ol>\n<li dir=\"ltr\">The number of samples for each Class (target variable) value.<\/li>\n<li dir=\"ltr\">Evaluation metrics at both the decision tree and random forest classification models.<\/li>\n<\/ol>\n<p dir=\"ltr\" style=\"text-align: justify;\">Do you remember the number of samples\/rows for each target value?&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">No? okay, let us check that number.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">See the number of samples for <strong>Class-1 (fraudulent)<\/strong> less than the samples for <strong>class-0 (non-fraudulent)<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This kind of dataset is called <strong>unbalanced<\/strong> data. Which means one class label samples are &nbsp;higher and dominating the other class label.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For a balanced dataset, accuracy is suitable because we take the divided value of the correctly predicted samples count with the total number of samples for accuracy.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\"><em><strong>Accuracy = number of correctly predicted samples \/ total number of samples<\/strong><\/em><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For example.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If our dataset has 20 samples, out of that 2 for Class 0 &amp; 18 for Class 1. Our trained model correctly predicted 17 samples out of 18 Class-1 samples and 0 samples out of 2 Class-0 samples.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">What is the accuracy value for this? 85%.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">But this is not correct, right? Because the model doesn\u2019t even predict one sample correctly for Class-0 samples, but we got 85% accuracy.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For an unbalanced dataset, a list of evaluation metrics are available. In the next section, we will discuss this.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600823366414\">Suitable evaluation metrics for imbalanced data<\/h2>\n<p dir=\"ltr\">So which all metrics are suitable for unbalanced data?<\/p>\n<p><span><img decoding=\"async\" alt=\"Evaluation Metrics for imbalance data\" data-id=\"6174\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"Evaluation Metrics for imbalance data\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/9-Evaluation-Metrics-for-imbalance-data.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 613px; --smush-placeholder-aspect-ratio: 613\/368;\"><noscript><img decoding=\"async\" alt=\"Evaluation Metrics for imbalance data\" data-id=\"6174\" width=\"613\" data-init-width=\"750\" height=\"368\" data-init-height=\"450\" title=\"Evaluation Metrics for imbalance data\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/9-Evaluation-Metrics-for-imbalance-data.png\" data-width=\"613\" data-height=\"368\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Evaluation Metrics for imbalance data<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can use any of the below-mentioned metrics for unbalanced or skewed datasets.<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Recall<\/li>\n<li dir=\"ltr\">Precision<\/li>\n<li dir=\"ltr\">F1-score<\/li>\n<li dir=\"ltr\">Area Under ROC curve.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can see the huge difference among different evaluation metrics for both classifications (decision tree &amp; random forest) models.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Do you remember we mentioned at model development stage, accuracy, classification report, etc. ?&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Okay, let see the results here.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600823366415\">Decision Tree Classification model results<\/h3>\n<\/p>\n<h3 id=\"t-1600823366416\">Random Forest Classification model results<\/h3>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Here we have to discuss a few terms and formulae related to confusion matrix, precision, recall &amp; F1-score.<\/p>\n<p><span><img decoding=\"async\" alt=\"Confusion matrix full representation\" data-id=\"4315\" width=\"626\" data-init-width=\"1024\" height=\"408\" data-init-height=\"667\" title=\"Confusion matrix full representation\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/12-Confusion-matrix-full-representation-1024x667.png\" data-width=\"626\" data-height=\"408\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/408;\"><noscript><img decoding=\"async\" alt=\"Confusion matrix full representation\" data-id=\"4315\" width=\"626\" data-init-width=\"1024\" height=\"408\" data-init-height=\"667\" title=\"Confusion matrix full representation\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/12-Confusion-matrix-full-representation-1024x667.png\" data-width=\"626\" data-height=\"408\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<ol>\n<li dir=\"ltr\"><strong>True Positive (TP):-<\/strong> &nbsp;<\/li>\n<\/ol>\n<p dir=\"ltr\">The number of positive labels correctly predicted by trained models.&nbsp; This means the number of Class-1 samples correctly predicted as Class-1.<\/p>\n<ol start=\"2\">\n<li dir=\"ltr\"><strong>True Negative (TN):- <\/strong><\/li>\n<\/ol>\n<p dir=\"ltr\">The number of negative labels correctly predicted by trained models.&nbsp; This means the number of Class-0 samples correctly predicted as Class-0.<\/p>\n<ol start=\"3\">\n<li dir=\"ltr\"><strong>False Positive (FP):- &nbsp;<\/strong><\/li>\n<\/ol>\n<p dir=\"ltr\">The number of positive labels incorrectly predicted by trained models. This means the number of Class-1 samples incorrectly predicted as Class-0.<\/p>\n<ol start=\"4\">\n<li dir=\"ltr\"><strong>False Negative (FN):- <\/strong>&nbsp;<\/li>\n<\/ol>\n<p dir=\"ltr\">The number of negative labels incorrectly predicted by trained models.&nbsp; This means the number of Class-0 samples incorrectly predicted as Class-1.<\/p>\n<h4 dir=\"ltr\">Formulae<\/h4>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Recall = TP \/ (TP + FN)<\/li>\n<li dir=\"ltr\">Precision = TP \/ (TP + FP)<\/li>\n<li dir=\"ltr\">F1-Score = 2*P*R \/ (P + R) here P for Precision, R for Recall<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Both classification models got accuracy scores as 99%.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">But when we observe the result of the classification report of both classifiers, f1-score for Class-0 got 100%, but for Class-1, F1-scores are significantly less.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">All these variations occur due to the unbalanced or skewed dataset.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Why f1-score for class-0 100%?&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Because of the number of samples for class-0 (2 lakhs). The number of samples for Class-0 is very high than the Class-1 samples.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">So what we need to do here is handle an unbalanced dataset. If you want to learn more about it, check the <a href=\"https:\/\/dataaspirant.com\/handle-imbalanced-data-machine-learning\/\" style=\"outline: none;\" target=\"_blank\">Best ways to handle unbalanced data in the machine learning<\/a> article which explained various ways to handle the imbalanced data.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">One more thing is left for discussion in this section, which is about areas under the ROC curve.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600823366417\">AUC and ROC Curves<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">Area Under ROC curve is another evaluation metric for classification problems. This is mostly suitable for skewed datasets. It tells us about model performance, such as the model&#8217;s capability to distinguish between target classes.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The effective model has a higher Area Under the <strong>ROC curve<\/strong> value. Here we measure the ability of class separability of a model by using the Area Under ROC curve.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Good models have AUC value <strong>near to 1<\/strong>, and the worst models have AUC value <strong>near 0.<\/strong><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">All the model performance methods help in the measuring the performance of the model based on the problem, but how to build the best models when we face with the data imbalance issue?<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For that, we need to apply different sampling methods to the data before building the models.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let\u2019s see how sampling methods improve model performance, and how much AUC score for that model in the coming section.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600823366418\">Model Improvement Using Sampling Techniques<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Data sampling is the statistical method for selecting data points (here, the data point is a single row) from the whole dataset. In machine learning problems, there are many sampling techniques available.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Here we take <strong>undersampling<\/strong> and oversampling strategies for handling imbalanced data. &nbsp;<\/p>\n<blockquote dir=\"ltr\" style=\"text-align: justify;\"><p><strong>What is this undersampling and oversampling?<\/strong><\/p><\/blockquote>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let us take an example of a dataset that has nine samples.&nbsp;<\/p>\n<ul>\n<li ltr\"\"=\"\">Six samples belong to class-0,<\/li>\n<li ltr\"\"=\"\">Three samples belong to class-1<\/li>\n<\/ul>\n<p ltr\"=\"\" style=\"text-align: justify;\" \"=\"\"><strong>Oversampling<\/strong> = 6 class-0 samples x &nbsp;2 times of class-1 samples of 3<\/p>\n<p ltr\"=\"\" style=\"text-align: justify;\" \"=\"\"><strong>Undersampling<\/strong> = 3 Class-1 samples x 3 samples from Class-0<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Here what we are trying to do is the number of samples of both target classes to be equal.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the oversampling technique, <strong>samples are repeated<\/strong>, and the dataset size is <strong>larger<\/strong> than the original dataset.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the undersampling technique, <strong>samples are not repeated<\/strong>, and the dataset size is <strong>less<\/strong> than the original dataset.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600823366419\">Applying Sampling Techniques&nbsp;<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">For undersampling techniques, we are checking the number of samples of both classes and selecting the smaller number and taking random samples from other class samples to create a new dataset. &nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The new dataset has an equal number of samples for both target classes.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This is a whole process of undersampling, and now we are going to implement this entire process using python.<\/p>\n<\/p>\n<p>The above is the target class distributions, now let&#8217;s see how we can change this.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Here first, we take <strong>indexes<\/strong> of both classes and randomly choose Class-0 samples indexes that are equal to the number of Class-1 samples.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the below code snippet, Combine both classes indexes. Then we <strong>extract<\/strong> all features of gathered indexes.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The above code first divides features and targets as <strong>x_undersample_data<\/strong> and <strong>y_undersample_data<\/strong> and then splits new undersample data into train and test dataset.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Okay, now we will call both classifiers with these new under sampling train and test datasets.<\/p>\n<h3 dir=\"ltr\" id=\"t-1600823366420\">Decision tree classification after applying sampling techniques<\/h3>\n<\/p>\n<p>Below are the model performance details<\/p>\n<\/p>\n<h3 id=\"t-1600823366421\">Random Forest Tree Classifier after applying the sampling techniques<\/h3>\n<\/p>\n<p>Below are the model performance details after applying the sampling techniques.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">See, the results of the F1-score for both target values are 95%, and the Area Under ROC curve is near to 1.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For the best models, we have the AUROC value near to 1. Here we implemented the undersampling technique; you can apply oversampling also like an undersampling process.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600823366422\">Conclusion<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Finally, our model gives 94% of the Area Under the ROC curve value. We can improve model results by adding more trees or applying additional data preprocessing techniques, etc.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Not only decision trees or random forest classifiers suitable for this problem. You can try with other machine learning classification algorithms such as Support Vector Machines (SVM), k-nearest neighbors, etc. &nbsp;to check how different algorithms are performed on classifying fraudulent activities.<\/p>\n<h2 dir=\"ltr\" id=\"t-1600823366423\">What next<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Try to use different classification algorithms to solve the same problem and check the F1 score for all the models. For <strong>implementation<\/strong>, you can have a look at the code snippets from the below articles.<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\"><strong><a href=\"https:\/\/dataaspirant.com\/support-vector-machine-classifier-implementation-r-caret-package\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Support Vector Machines implementation<\/a>&nbsp;<\/strong><\/li>\n<li dir=\"ltr\"><strong><a href=\"https:\/\/dataaspirant.com\/k-nearest-neighbor-algorithm-implementaion-python-scratch\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">K-nearest neighbors implementation<\/a><\/strong><\/li>\n<li dir=\"ltr\"><strong><a href=\"https:\/\/dataaspirant.com\/gaussian-naive-bayes-classifier-implementation-python\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Navie bias algorithm &nbsp;implementation<\/a><\/strong><\/li>\n<li dir=\"ltr\"><strong><a href=\"https:\/\/dataaspirant.com\/implement-logistic-regression-model-python-binary-classification\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Logistic Regression algorithm implementation<\/a><\/strong><\/li>\n<\/ul>\n<h4>Recommended Courses<\/h4>\n<p>Recommended<span style=\"\"><img decoding=\"async\" alt=\"credit risk modeling\" data-id=\"6246\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"credit risk modeling\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/credit-risk-modeling-150x150.jpg\" data-width=\"176\" data-height=\"176\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 176px; --smush-placeholder-aspect-ratio: 176\/176;\"><noscript><img decoding=\"async\" alt=\"credit risk modeling\" data-id=\"6246\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"credit risk modeling\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/credit-risk-modeling-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Credit Risk modelling in Python<\/h4>\n<p style=\"\">Rating: <strong>4\/5<\/strong><\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"data science bootcamp\" data-id=\"6247\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"data science bootcamp\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/data-science-bootcamp-150x150.jpg\" data-width=\"176\" data-height=\"176\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 176px; --smush-placeholder-aspect-ratio: 176\/176;\"><noscript><img decoding=\"async\" alt=\"data science bootcamp\" data-id=\"6247\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"data science bootcamp\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/09\/data-science-bootcamp-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Data Science Bootcamp Course<\/h4>\n<p style=\"\">Rating: <strong>4.5\/5<\/strong><\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"Machine learning\" data-id=\"4302\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"machine learning\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/machine-learning-1-150x150.jpg\" data-width=\"176\" data-height=\"176\" mt-d=\"-19\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" style=\"--smush-placeholder-width: 176px; --smush-placeholder-aspect-ratio: 176\/176;\"><noscript><img decoding=\"async\" alt=\"Machine learning\" data-id=\"4302\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"machine learning\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/08\/machine-learning-1-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"\" mt-d=\"-19\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Machine Learning A to Z Course<\/h4>\n<p style=\"\">Rating: <strong>4.5\/5<\/strong><\/p>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>|\u00a0<span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span>\u00a0|\u00a0<span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span>\u00a0|\u00a0<span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0|\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions ? or want me to write an article on a specific topic? then feel free to comment below.<\/p>\n<p>[tcb-script type=&#8221;text\/javascript&#8221;] !function(w,d,a){if(!w[a]){var s=d.createElement(&#8216;script&#8217;), x=d.getElementsByTagName(&#8216;script&#8217;)[0];s.type=&#8217;text\/javascript&#8217;; s.async=!0;s.src =&#8217;\/\/widgets.ad.style\/native.js&#8217;;x.parentNode .insertBefore(s,x);}w[a]=w[a]||function(){(w[a].q=w[a].q||[]) .push(arguments)};}(window, document, &#8216;_as&#8217;);_as(&#8216;create&#8217;,{wi:2820});[\/tcb-script]<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Credit Card Fraud Detection With Classification Algorithms In Python Fraud transactions or fraudulent activities are significant issues in many industries like banking, insurance, etc. Especially for the banking industry, credit card fraud detection is a pressing issue to resolve. These industries suffer too much due to fraudulent activities towards revenue growth and lose customer\u2019s trust.<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/credit-card-fraud-detection-classification-algorithms-python\/\">+ Read More<\/a><\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[6,2103],"tags":[],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/6113"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=6113"}],"version-history":[{"count":4,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/6113\/revisions"}],"predecessor-version":[{"id":9933,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/6113\/revisions\/9933"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=6113"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=6113"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=6113"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}