{"id":7611,"date":"2020-12-03T19:06:04","date_gmt":"2020-12-03T19:06:04","guid":{"rendered":"https:\/\/dataaspirant.com\/?p=7611"},"modified":"2023-10-19T19:56:11","modified_gmt":"2023-10-19T14:26:11","slug":"cross-validation","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/cross-validation\/","title":{"rendered":"How Cross-Validation Works In Machine Learning"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><p><span><img decoding=\"async\" alt=\"Cross Validation\" data-id=\"7615\" width=\"596\" data-init-width=\"750\" height=\"358\" data-init-height=\"450\" title=\"Cross Validation\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/1-Cross-Validation.png\" data-width=\"596\" data-height=\"358\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/358;aspect-ratio: auto 750 \/ 450;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Cross Validation\" data-id=\"7615\" width=\"596\" data-init-width=\"750\" height=\"358\" data-init-height=\"450\" title=\"Cross Validation\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/1-Cross-Validation.png\" data-width=\"596\" data-height=\"358\" style=\"aspect-ratio: auto 750 \/ 450;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p>Cross validation defined as:<\/p>\n<blockquote dir=\"ltr\" style=\"text-align: justify;\"><p><strong>\u201cA statistical method or a resampling procedure used to evaluate the skill of machine learning models on a limited data sample.\u201d<\/strong><\/p><\/blockquote>\n<p dir=\"ltr\" style=\"text-align: justify;\">It is mostly used while building <a href=\"https:\/\/dataaspirant.com\/credit-card-fraud-detection-classification-algorithms-python\/\" target=\"_blank\" rel=\"noopener\"><strong>machine learning models<\/strong><\/a>. It compares and selects a model for a given predictive modeling problem, assesses the models\u2019 predictive performance.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Later judges how they perform outside to a new data set, also known as <strong>test data<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The motivation to use cross validation techniques is that we are <strong>holding<\/strong> it to a training dataset when we fit a model.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In this tutorial, along with cross validation we will also have a soft focus on the <strong>k-fold cross-validation<\/strong> procedure for <a href=\"https:\/\/dataaspirant.com\/six-popular-classification-evaluation-metrics-in-machine-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>evaluating the performance<\/strong><\/a> of the machine learning models.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Toward the end of this instructional exercise, you will become more acquainted with the below topics:<\/p>\n<ul>\n<li dir=\"ltr\"><strong>Various<\/strong> types of cross-validation among which k-fold cross-validation is the most commonly used<\/li>\n<li dir=\"ltr\">K-fold cross-validation is a resampling procedure that estimates the skill of the machine learning model on new data.<\/li>\n<li dir=\"ltr\">Some common strategies that we can use to <strong>select the value of k<\/strong> for our dataset<\/li>\n<li dir=\"ltr\">Common variations in cross-validation such as stratified and repeated that are available in scikit-learn.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Before we start learning, Let\u2019s have a look at the topics you will learn in this article. Only if you read the complete article \ud83d\ude42<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">To learn the cross validation topic, you need to know about the overfitting and underfitting. No need to know <a href=\"https:\/\/dataaspirant.com\/handle-overfitting-deep-learning-models\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>how to handle overfitting<\/strong><\/a> but at least the issue.<\/p>\n<h2 dir=\"ltr\" id=\"t-1607016614325\">Concept Of Model Underfitting &amp; Overfitting<\/h2>\n<p><span><img decoding=\"async\" alt=\"Underfitting and Overfitting\" data-id=\"7622\" width=\"596\" data-init-width=\"750\" height=\"358\" data-init-height=\"450\" title=\"Underfitting and Overfitting\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/2-Underfitting-and-Overfitting.png\" data-width=\"596\" data-height=\"358\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/358;aspect-ratio: auto 750 \/ 450;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Underfitting and Overfitting\" data-id=\"7622\" width=\"596\" data-init-width=\"750\" height=\"358\" data-init-height=\"450\" title=\"Underfitting and Overfitting\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/2-Underfitting-and-Overfitting.png\" data-width=\"596\" data-height=\"358\" style=\"aspect-ratio: auto 750 \/ 450;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Whenever a statistical model or a <a href=\"https:\/\/dataaspirant.com\/classification-clustering-alogrithms\/\" target=\"_blank\" rel=\"noopener\"><strong>machine learning algorithm<\/strong><\/a> captures the data\u2019s noise, underfitting comes into play.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Intuitively, overfitting occurs when the machine learning algorithm or the model <strong>fits the data too well<\/strong>. Whenever overfitting occurs, the model gives a good performance and accuracy on the training data set but a <strong>low accuracy<\/strong> on new <strong>unseen<\/strong> data sets.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Contrary to that, whenever a statistical model or a machine learning algorithm cannot capture the data\u2019s underlying trends, <strong>under-fitting<\/strong> comes into play.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Intuitively, under-fitting occurs when the the model does not fit the information <strong>well enough<\/strong>. It can be said that under-fitting is a consequence of a straightforward model.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The term <strong>\u201csimple\u201d<\/strong> means the underlying missing data of the model is not adequately handled. The <a href=\"https:\/\/dataaspirant.com\/feature-selection-techniques-r\/\" target=\"_blank\" rel=\"noopener\"><strong>irrelevant features<\/strong><\/a> that do not contribute much to the predictor variable are not removed.<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614326\">How Can We Prevent Model Overfitting<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">In machine learning, a significant challenge with overfitting is that we are unaware of how our model will perform on the new data (test data) until we test it ourselves.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Generally we split our initial dataset into <strong>two subsets<\/strong>, i-e, training, and test subsets, to address this issue.<\/p>\n<p><span><img decoding=\"async\" alt=\"Underfitting Overfitting\" data-id=\"7626\" width=\"596\" data-init-width=\"1024\" height=\"240\" data-init-height=\"412\" title=\"Underfitting Overfitting\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/3-Underfitting-Overfitting-1024x412.png\" data-width=\"596\" data-height=\"240\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/240;aspect-ratio: auto 1024 \/ 412;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Underfitting Overfitting\" data-id=\"7626\" width=\"596\" data-init-width=\"1024\" height=\"240\" data-init-height=\"412\" title=\"Underfitting Overfitting\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/3-Underfitting-Overfitting-1024x412.png\" data-width=\"596\" data-height=\"240\" style=\"aspect-ratio: auto 1024 \/ 412;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If we use a smart way to use the available initial dataset to multiple test datasets, we can <a href=\"https:\/\/dataaspirant.com\/handle-overfitting-deep-learning-models\/\" target=\"_blank\" rel=\"noopener\"><strong>overcome the issue of overfitting<\/strong><\/a>. As now our model learns on various train datasets.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This smart is nothing but <strong>cross validation<\/strong>. This makes more sense, when we explain how we can create multiple train datasets in the upcoming sections of this article.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">&nbsp;Now comes the real question.<\/p>\n<h2 dir=\"ltr\" id=\"t-1607016614327\">What Is Cross Validation?<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Cross-validation is the best preventive measure against overfitting. It is a smart technique that allows us to utilize our data in a better way.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the data mining models or machine learning models, separation of data into training and testing sets is an essential part. Minimizing the data discrepancies and better understanding of the machine learning model\u2019s properties can be done using similar data for the training and testing subsets.<\/p>\n<blockquote dir=\"ltr\" style=\"text-align: justify;\"><p>How about using the same data, if we create <strong>multiple training<\/strong> datasets ?&nbsp;<\/p><\/blockquote>\n<p dir=\"ltr\" style=\"text-align: justify;\">Where come cross validation.<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614328\">Cross Validation Procedure<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">There are <strong>different types<\/strong> or variations of cross-validation, but the overall procedure remains the same.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The following is the procedure deployed in almost all types of cross-validation:<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">&nbsp;<strong>Partition or divide<\/strong> the dataset into several subsets<\/li>\n<li dir=\"ltr\">&nbsp;At one time, keep or hold out one of the set and train the model on the remaining set<\/li>\n<li dir=\"ltr\">&nbsp;Perform the model testing on the holdout dataset<\/li>\n<\/ul>\n<p dir=\"ltr\">The same procedure is <strong>repeated<\/strong> for each subset of the dataset.<\/p>\n<p><span><img decoding=\"async\" alt=\"Cross Validation Procedure\" data-id=\"7629\" width=\"596\" data-init-width=\"2540\" height=\"196\" data-init-height=\"834\" title=\"Cross Validation Procedure\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/4-Cross-Validation-Procedure.png\" data-width=\"596\" data-height=\"196\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/196;aspect-ratio: auto 2540 \/ 834;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Cross Validation Procedure\" data-id=\"7629\" width=\"596\" data-init-width=\"2540\" height=\"196\" data-init-height=\"834\" title=\"Cross Validation Procedure\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/4-Cross-Validation-Procedure.png\" data-width=\"596\" data-height=\"196\" style=\"aspect-ratio: auto 2540 \/ 834;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Sometimes, the data splitting is done into training and validation\/test sets when building a machine learning model. The main reason for the training set is to fit the model, and the purpose of the validation\/test set is to validate\/test it on new data that it has never seen before.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can do a classic <strong>80-20%<\/strong> split, but different values such as 70%-30% or 90%-10% can also be used depending on the dataset\u2019s size.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">It is challenging to evaluate and make changes in the model that outweigh our data. So we create two sections of our data as under<\/p>\n<p><span><img decoding=\"async\" alt=\"Training And Testing\" data-id=\"7632\" width=\"596\" data-init-width=\"750\" height=\"358\" data-init-height=\"450\" title=\"Training And Testing\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/5-Training-And-Testing.png\" data-width=\"596\" data-height=\"358\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/358;aspect-ratio: auto 750 \/ 450;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Training And Testing\" data-id=\"7632\" width=\"596\" data-init-width=\"750\" height=\"358\" data-init-height=\"450\" title=\"Training And Testing\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/5-Training-And-Testing.png\" data-width=\"596\" data-height=\"358\" style=\"aspect-ratio: auto 750 \/ 450;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Training Data<\/li>\n<li dir=\"ltr\">Test Data<\/li>\n<\/ul>\n<h3 dir=\"ltr\" id=\"t-1607016614329\">Training Data<\/h3>\n<p dir=\"ltr\">&nbsp;Training data can be defined as:<\/p>\n<blockquote dir=\"ltr\"><p>&nbsp; &nbsp; &nbsp; &nbsp;<strong> \u201cThe data used to construct the model.\u201d<\/strong><\/p><\/blockquote>\n<p dir=\"ltr\">Most of our data should be used as training data as it provides insight into the relationship between our given inputs.&nbsp;<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614330\">&nbsp;Test Data<\/h3>\n<p dir=\"ltr\">Test data can be defined as:<\/p>\n<blockquote dir=\"ltr\"><p>&nbsp; &nbsp; &nbsp; &nbsp;<strong> \u201cThe data used for model validation\/testing.\u201d<\/strong><\/p><\/blockquote>\n<p dir=\"ltr\" style=\"text-align: justify;\">The models generated are to predict the results unknown, which is named as the test set.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can use test data on our model to see how well our model performs on data it has never seen before.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Depending upon the <a href=\"https:\/\/dataaspirant.com\/six-popular-classification-evaluation-metrics-in-machine-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>performance of our model<\/strong><\/a> on our test data, we can make adjustments to our model, such as mentioned below:<\/p>\n<ul>\n<li dir=\"ltr\">&nbsp; Change the values of the <strong>hyper-parameters<\/strong> i-e; <strong>\u03b1 and \u03bb<\/strong> , also know as <a href=\"https:\/\/dataaspirant.com\/hyperparameter-tuning-with-keras-tuner\/\" target=\"_blank\" rel=\"noopener\"><strong>hyper parameter tuning<\/strong><\/a>.<\/li>\n<li dir=\"ltr\">&nbsp;&nbsp;Adjust the number of variables in the model<\/li>\n<li dir=\"ltr\">&nbsp; Change the number of layers in the <a href=\"https:\/\/dataaspirant.com\/neural-network-basics\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>neural network<\/strong><\/a><\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now we get a more refined definition of cross-validation, which is as:<\/p>\n<blockquote dir=\"ltr\" style=\"text-align: justify;\"><p><strong>\u201cThe process of using test data to assess the model\u2019s performance is termed as cross-validation.\u201d<\/strong><\/p><\/blockquote>\n<h2 dir=\"ltr\" id=\"t-1607016614331\">Variations on Cross Validation<\/h2>\n<p dir=\"ltr\">The commonly used variations on cross-validation are discussed below:<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614332\">Train\/Test Split<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">The train-test split evaluates the performance and the skill of the machine learning algorithms when they make predictions on the data not used for model training.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">It is an easy and fast procedure to implement as the results allow us to compare our algorithms\u2019 performance for the predictive modeling problem. Though the method is simple and easy to use, some scenarios do not work well.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For example, in small datasets and the situation in which additional configuration is needed, the method does not work well.<\/p>\n<p><span><img decoding=\"async\" alt=\"Train And Test Datasets\" data-id=\"7636\" width=\"554\" data-init-width=\"1024\" height=\"310\" data-init-height=\"573\" title=\"Train And Test Datasets\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/6-Train-And-Test-Datasets-1024x573.png\" data-width=\"554\" data-height=\"310\" style=\"--smush-placeholder-width: 554px; --smush-placeholder-aspect-ratio: 554\/310;aspect-ratio: auto 1024 \/ 573;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Train And Test Datasets\" data-id=\"7636\" width=\"554\" data-init-width=\"1024\" height=\"310\" data-init-height=\"573\" title=\"Train And Test Datasets\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/6-Train-And-Test-Datasets-1024x573.png\" data-width=\"554\" data-height=\"310\" style=\"aspect-ratio: auto 1024 \/ 573;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614333\">Leave One Out Cross Validation (LOOCV)<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">This variation on cross-validation leaves one data point out of the training data. For instance, if there are n data points in the original data sample, then the pieces used to train the model are<strong> n-1<\/strong>, and <strong>p points<\/strong> will be used as the validation set.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This cycle is repeated in all of the combinations where the original sample can be separated in such a way. After this, the mean of the error is taken for all trials to give overall effectiveness.&nbsp;<\/p>\n<p>We consider that the number of possible combinations is equal to the number of data points in the original sample represented by <strong>n.<\/strong><span><img decoding=\"async\" alt=\"LOOCV Leave One Out Cross Validation\" data-id=\"7639\" width=\"626\" data-init-width=\"1024\" height=\"470\" data-init-height=\"769\" title=\"LOOCV Leave One Out Cross Validation\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/7-LOOCV-Leave-One-Out-Cross-Validation-1024x769.png\" data-width=\"626\" data-height=\"470\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/470;aspect-ratio: auto 1024 \/ 769;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"LOOCV Leave One Out Cross Validation\" data-id=\"7639\" width=\"626\" data-init-width=\"1024\" height=\"470\" data-init-height=\"769\" title=\"LOOCV Leave One Out Cross Validation\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/7-LOOCV-Leave-One-Out-Cross-Validation-1024x769.png\" data-width=\"626\" data-height=\"470\" style=\"aspect-ratio: auto 1024 \/ 769;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614334\">Stratified<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">The process of rearranging the data to ensure that each fold is a good representative of the whole is termed stratification.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For instance, in the case of a <a href=\"https:\/\/dataaspirant.com\/implement-logistic-regression-model-python-binary-classification\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>binary classification problem<\/strong><\/a>, each class is comprises of&nbsp;<strong>50%<\/strong> of the data.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let&#8217;s say the ration is <strong>30% and 70%<\/strong> distribution. So the best practice is to arrange the data so that each class consists of the same 30% and 70% distribution in every fold.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\"><strong>Note<\/strong> that 30% and 70% ration is not <a href=\"https:\/\/dataaspirant.com\/handle-imbalanced-data-machine-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>imbalanced data<\/strong><\/a>.<\/p>\n<p><span><img decoding=\"async\" alt=\"Stratified K-Fold Cross Validation\" data-id=\"7642\" width=\"596\" data-init-width=\"2114\" height=\"401\" data-init-height=\"1420\" title=\"Stratified K-Fold Cross Validation\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/8-Stratified-K-Fold-Cross-Validation.png\" data-width=\"596\" data-height=\"401\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/401;aspect-ratio: auto 2114 \/ 1420;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Stratified K-Fold Cross Validation\" data-id=\"7642\" width=\"596\" data-init-width=\"2114\" height=\"401\" data-init-height=\"1420\" title=\"Stratified K-Fold Cross Validation\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/8-Stratified-K-Fold-Cross-Validation.png\" data-width=\"596\" data-height=\"401\" style=\"aspect-ratio: auto 2114 \/ 1420;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614335\">Repeated<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">&nbsp;In this method, the k-fold cross-validation method undergoes <strong>n number of repetitions.<\/strong><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">More importantly, the data sample\u2019s <strong>shuffling<\/strong> is done before each repetition, resulting in a different sample split.<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614336\">Nested<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">In this method, the k-fold cross-validation is performed within each fold of cross-validation, Sometimes to perform <a href=\"https:\/\/dataaspirant.com\/hyperparameter-tuning-with-keras-tuner\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>tuning of the hyperparameters<\/strong><\/a> during the evaluation of the machine learning model.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This process is termed nested or <strong>double cross-validation<\/strong>.&nbsp;<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614337\">Train \u2013 Test Split<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">This method usually split our data into the <strong>80:20<\/strong> ratio between the training and test data.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The technique works well enough when the amount of data is large, say when we have 1000+ rows of data.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">1. When we are working with <strong>100,000+<\/strong> rows of data, the ratio of 90:10 can be of use, and with 1, 00,000+ data rows, we can use a 99:1 balance.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">2. Generally, when working with a large amount of data. A <strong>smaller percentage<\/strong> of test data can be used since the amount of the training data is sufficient to build a <a href=\"https:\/\/dataaspirant.com\/decision-tree-algorithm-python-with-scikit-learn\/\" target=\"_blank\" rel=\"noopener\"><strong>reasonably accurate model<\/strong><\/a>.<\/p>\n<h4 dir=\"ltr\">Cost Function<\/h4>\n<p><span><img decoding=\"async\" alt=\"Cost function\" data-id=\"7645\" width=\"497\" data-init-width=\"497\" height=\"102\" data-init-height=\"102\" title=\"Cost function\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/9-cost-function.png\" data-width=\"497\" data-height=\"102\" style=\"--smush-placeholder-width: 497px; --smush-placeholder-aspect-ratio: 497\/102;aspect-ratio: auto 497 \/ 102;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Cost function\" data-id=\"7645\" width=\"497\" data-init-width=\"497\" height=\"102\" data-init-height=\"102\" title=\"Cost function\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/9-cost-function.png\" data-width=\"497\" data-height=\"102\" style=\"aspect-ratio: auto 497 \/ 102;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let\u2019s have a look at the cost function or mean squared error of our test data. These kind of cost functions help in <a href=\"https:\/\/dataaspirant.com\/optimization-algorithms-deep-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>optimizing the errors<\/strong><\/a> the model made.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the above formula, <strong>m_test<\/strong> shows the number of training examples in test data.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">To assess the execution of our model, we can make adjustments accordingly. <strong><a href=\"https:\/\/dataaspirant.com\/six-popular-classification-evaluation-metrics-in-machine-learning\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">several evaluation metrics<\/a><\/strong> are there.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Such as,<\/p>\n<ul>\n<li dir=\"ltr\" style=\"text-align: justify;\">Mean Square Error (MSE),&nbsp;<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\">Root Mean Square Error (RMSE),<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\"><strong><a href=\"https:\/\/dataaspirant.com\/difference-between-r-squared-and-adjusted-r-squared\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">R-Squared and Adjusted R-Squared methods.<\/a><\/strong><\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">&nbsp;The above mentioned metrics are for <a href=\"https:\/\/dataaspirant.com\/linear-regression\/\" target=\"_blank\" rel=\"noopener\"><strong>regression<\/strong><\/a> kind of problems.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Below are the advantages and disadvantages of the Train \u2013 Test Split method.<\/p>\n<h4 style=\"text-align: center;\">Advantages<\/h4>\n<ul>\n<li style=\"\"><span>Train \u2013 Test Split works well with large data sets.<\/span><\/li>\n<li style=\"\"><span>The computing power is low.<\/span><\/li>\n<li style=\"\"><span>The&nbsp;feedback for model performance can be obtained quickly.<\/span><\/li>\n<\/ul>\n<h4 style=\"text-align: center;\">Disadvantages<\/h4>\n<ul>\n<li style=\"\">                    <span>Train \u2013 Test Split works very poorly on small data sets.<\/span><\/li>\n<li style=\"\">                    <span>It often leads to the development of the models having high bias when working on small data sets.<\/span><\/li>\n<li style=\"\">                    <span>There is a possibility of selecting test data with similar values, i-e, non-random values, resulting in an inaccurate evaluation of model performance.<\/span><\/li>\n<\/ul>\n<h2 dir=\"ltr\" id=\"t-1607016614338\">K-Fold Cross Validation<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">In k-fold cross-validation, we do more than one split. We can do 3, 5, 10, or any K number of partitions.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">These splits are called <strong>folds<\/strong>, and the method works well by splitting the data into folds, usually consisting of around <strong>10-20%<\/strong> of the data.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Upon each iteration, we use different training folds to construct our model; therefore, the parameters which are produced in each model may differ slightly.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The <strong>model parameters<\/strong> generated in each case are also averaged to make a final model.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The k-fold procedure has a single parameter termed k, which depicts the number of groups the sample data can be split into.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For example, if we set the value <strong>k=5<\/strong>, the dataset will be divided into five equal parts.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Following the general cross-validation procedure, the process will run five times, each time with a different holdout set.<\/p>\n<p><span><img decoding=\"async\" alt=\"K-Fold Cross Validation\" data-id=\"7653\" width=\"596\" data-init-width=\"1524\" height=\"499\" data-init-height=\"1274\" title=\"K-Fold Cross Validation\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/10-K-Fold-Cross-Validation.png\" data-width=\"596\" data-height=\"499\" style=\"--smush-placeholder-width: 596px; --smush-placeholder-aspect-ratio: 596\/499;aspect-ratio: auto 1524 \/ 1274;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"K-Fold Cross Validation\" data-id=\"7653\" width=\"596\" data-init-width=\"1524\" height=\"499\" data-init-height=\"1274\" title=\"K-Fold Cross Validation\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/10-K-Fold-Cross-Validation.png\" data-width=\"596\" data-height=\"499\" style=\"aspect-ratio: auto 1524 \/ 1274;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p>Below are the advantages and disadvantages of k-fold cross-validation.<\/p>\n<h4 id=\"t-1607016614339\" style=\"text-align: center;\">Advantages<\/h4>\n<ul>\n<li style=\"\"><span>K-fold cross-validation works well on small and large data sets.<\/span><\/li>\n<li style=\"\"><span>All of our data is used in testing our model, thus giving a fair, well-rounded evaluation metric.<\/span><\/li>\n<li style=\"\"><span>K-fold cross-validation may lead to more accurate models since we are eventually utilizing our data to build our model.<\/span><\/li>\n<\/ul>\n<h4 id=\"t-1607016614340\" style=\"text-align: center;\">Disadvantages<\/h4>\n<ul>\n<li style=\"\">                    <span>The computing power is high.<\/span><\/li>\n<li style=\"\">                    <span>So it may take some time to get feedback on the model\u2019s performance in the case of large data sets.<\/span><\/li>\n<li style=\"\">                    <span>Slower feedback makes it take longer to find the optimal hyperparameters for the model.<\/span><\/li>\n<\/ul>\n<h2 dir=\"ltr\" id=\"t-1607016614341\">Configuration Of K in Cross Validation<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Now, let\u2019s discuss how we can select <strong>the value of k<\/strong> for our data sample.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">It must be noted that the value of k must be chosen carefully because a poorly chosen value for k may give a vague idea of the machine learning model\u2019s skill.<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614342\">Common tactics for choosing the value of k<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">The common strategies for choosing a value for k are as under<\/p>\n<h4 dir=\"ltr\" style=\"text-align: justify;\"><strong>Representative:<\/strong><\/h4>\n<p dir=\"ltr\" style=\"text-align: justify;\">We chose the value of k so that each train\/test subset of the data sample is large enough to be a statistical representation of the broader dataset.<\/p>\n<h4 dir=\"ltr\" style=\"text-align: justify;\"><strong>Using the value k=10:&nbsp;<\/strong><\/h4>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the field of applied machine learning, the most common value of k found through experimentation is k = 10, which generally results in a model skill estimate with low bias and a moderate variance.<\/p>\n<h4 dir=\"ltr\" style=\"text-align: justify;\"><strong>Putting k=n:<\/strong><\/h4>\n<p dir=\"ltr\" style=\"text-align: justify;\">In this strategy, the value for k is fixed to n, where n represents the dataset\u2019s size to allow each test sample to be used in the holdout dataset. This approach is called <strong>leave-one-out cross-validation<\/strong> (LOOCV).<\/p>\n<h3 dir=\"ltr\" id=\"t-1607016614343\">Deductions<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">Some essential deductions from the above strategies are as under:<\/p>\n<ol>\n<li dir=\"ltr\">\n<p dir=\"ltr\" style=\"text-align: justify;\">We usually use the value of<strong> k<\/strong>, either 5 or 10, but there is no hard and fast rule. When we use a considerable value of k, the difference between the training and the resampling subsets gets smaller. The bias gets smaller as the difference decreases.<\/p>\n<\/li>\n<li dir=\"ltr\">\n<p dir=\"ltr\" style=\"text-align: justify;\">Most commonly, the value of k=10 is used in the field of applied machine learning. A <a href=\"https:\/\/dataaspirant.com\/bias-variance-tradeoff\/\" target=\"_blank\" rel=\"noopener\"><strong>bias-variance tradeoff<\/strong><\/a> exists with the choice of k in k-fold cross-validation. Given this scenario, k-fold cross-validation can be performed using either k = 5 or k = 10, as these two values do not suffer from high bias and high variance.&nbsp;<\/p>\n<\/li>\n<li dir=\"ltr\">\n<p dir=\"ltr\" style=\"text-align: justify;\">When we choose a value of k that does not perform even splitting of the data, then the remainder of examples will be found in one group. We prefer to split our data sample into k number of groups having the same number of samples.<\/p>\n<\/li>\n<\/ol>\n<h2 dir=\"ltr\" id=\"t-1607016614344\">Example Depicting The K-Fold Procedure<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">In order to have a concrete concept of k-fold cross-validation, let have a look at the following example depicting its procedure.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We consider that we have <strong>6 observations<\/strong> as below:<\/p>\n<p><span><img decoding=\"async\" alt=\"K-Fold Cross Validation Example\" data-id=\"7663\" width=\"626\" data-init-width=\"1506\" height=\"448\" data-init-height=\"1078\" title=\"K-Fold Cross Validation Example\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/11-K-Fold-Cross-Validation-Example.png\" data-width=\"626\" data-height=\"448\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/448;aspect-ratio: auto 1506 \/ 1078;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"K-Fold Cross Validation Example\" data-id=\"7663\" width=\"626\" data-init-width=\"1506\" height=\"448\" data-init-height=\"1078\" title=\"K-Fold Cross Validation Example\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/11-K-Fold-Cross-Validation-Example.png\" data-width=\"626\" data-height=\"448\" style=\"aspect-ratio: auto 1506 \/ 1078;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Initially, the value of k is chosen to determine the number of folds required for splitting the data so that we will use a value of k=3.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">That means that first, we will shuffle the data and then split the data into three groups. As we have six observations, so each group will have an equal number of 2 observations.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The use of the sample can be made to evaluate the machine learning model\u2019s skill and performance.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Training and evaluation of three models are performed where each fold is allowed to be a held-out test set.<\/p>\n<p dir=\"ltr\">For example:<\/p>\n<ul>\n<li dir=\"ltr\"><strong>Model1:<\/strong> Training on fold1 and fold2, Testing on fold3<\/li>\n<li dir=\"ltr\"><strong>Model2:<\/strong> Training on fold2 and fold3, Testing on fold1<\/li>\n<li dir=\"ltr\"><strong>Model3:<\/strong> Training on fold1 and fold3, Testing on fold2<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">After the evaluation process ends, the models are discarded as their purpose has been served. The skill scores are then collected for each model and summarized for use.<\/p>\n<h2 dir=\"ltr\" id=\"t-1607016614345\">Cross Validation API<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">The k-fold cross-validation process needs not to be implemented manually. Implementation is provided by the <strong>scikit-learn<\/strong> library, which performs the splitting of the given data sample.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can use the &nbsp;<strong>KFold() scikit-learn<\/strong> class. It takes the number of splits as the arguments without taking into consideration whether the sampling of the data is done or not.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">An instance can be created that will perform the splitting of the dataset into three folds, performs shuffling of the data sample before the split. Then uses a value of 1 for the pseudorandom number generator.<\/p>\n<p>kfold = KFold(3, True, 1)<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">We can call the <strong>split() function<\/strong> on the class where the data sample is provided as an argument. The split() function will return each group of train and test sets on repeated calls.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In particular, the arrays containing the <strong>indexes<\/strong> are returned into the original data sample of observations to be further used for train and test sets on each iteration.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For example, the splits of the indices for the data sample can be enumerated using the created <strong>KFold<\/strong> instance, as shown below in the following code:<\/p>\n<p style=\"text-align: justify;\">All of this can be tied together with the small dataset mentioned above in the worked example.<\/p>\n<p style=\"text-align: justify;\">When we run the above example, specific observations chosen for each train and test set are printed.<\/p>\n<p style=\"text-align: justify;\">On the original data array, the indices are used directly to retrieve the observation values.<\/p>\n<p><span><img decoding=\"async\" alt=\"Sklearn Cross Validation Output\" data-id=\"7672\" width=\"626\" data-init-width=\"1024\" height=\"227\" data-init-height=\"372\" title=\"Sklearn Cross Validation Output\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/12-Sklearn-Cross-Validation-Output-1024x372.png\" data-width=\"626\" data-height=\"227\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/227;aspect-ratio: auto 1024 \/ 372;\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Sklearn Cross Validation Output\" data-id=\"7672\" width=\"626\" data-init-width=\"1024\" height=\"227\" data-init-height=\"372\" title=\"Sklearn Cross Validation Output\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/12-Sklearn-Cross-Validation-Output-1024x372.png\" data-width=\"626\" data-height=\"227\" style=\"aspect-ratio: auto 1024 \/ 372;\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the scikit-learn library, the k-fold cross validation implementation is provided as a component operation with broader methods such as scoring a given data sample model.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In scikit-learn, the k-fold cross-validation is provided as a component operation within more general practices, such as achieving a model on a dataset.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">However, we can use the K-Fold class directly for splitting up the dataset before modeling such that all of the <a href=\"https:\/\/dataaspirant.com\/support-vector-machine-classifier-implementation-r-caret-package\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>machine learning models<\/strong><\/a> will use the same splits of the data. This technique is mostly helpful when we are working with large datasets.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Using the same partitions of data across algorithms can have a lot of benefits for <a href=\"https:\/\/dataaspirant.com\/jarque-bera-test\/\" target=\"_blank\">statistical tests<\/a>.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">You can clone this article related codes in our <a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/tree\/master\/cross_validation\" target=\"_blank\" rel=\"noopener\"><strong>Github repo<\/strong><\/a>.<\/p>\n<h2 dir=\"ltr\" id=\"t-1607016614346\">Conclusion<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">So far, we have learned that a cross-validation is a powerful tool and a strong preventive measure against <a href=\"https:\/\/dataaspirant.com\/handle-overfitting-deep-learning-models\/\" target=\"_blank\" rel=\"noopener\"><strong>model overfitting<\/strong><\/a>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">With cross validation, we can better use our data and the excellent know-how of our algorithm\u2019s performance. Furthermore, we had a look at variations of cross-validation like LOOCV, stratified, k-fold, and so on.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In complicated <strong><a href=\"https:\/\/dataaspirant.com\/implement-multinomial-logistic-regression-python\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">machine learning models<\/a><\/strong>, sometimes it becomes a bit easy not paying attention and using the same sample data in different pipeline stages.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The consequence is that it may lead to good but not a real performance in most cases as strange side effects may be introduced.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Cross-validation can be of great use while dealing with the non-trivial challenges in the <a href=\"https:\/\/dataaspirant.com\/category\/projects\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>Data Science projects<\/strong><\/a>.<\/p>\n<h4>Recommended Courses<\/h4>\n<p>Recommended<span style=\"\"><img decoding=\"async\" alt=\"Machine Learning Courses\" data-id=\"9051\" width=\"191\" data-init-width=\"200\" height=\"191\" data-init-height=\"200\" title=\"Machine Learning Courses\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2023\/02\/1-1.png\" data-width=\"191\" data-height=\"191\" style=\"--smush-placeholder-width: 191px; --smush-placeholder-aspect-ratio: 191\/191;aspect-ratio: auto 200 \/ 200;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Machine Learning Courses\" data-id=\"9051\" width=\"191\" data-init-width=\"200\" height=\"191\" data-init-height=\"200\" title=\"Machine Learning Courses\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2023\/02\/1-1.png\" data-width=\"191\" data-height=\"191\" style=\"aspect-ratio: auto 200 \/ 200;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Machine Learning Course<\/h4>\n<p style=\"\">Rating: <strong>4.7\/5<\/strong><\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"Deep Learning Courses\" data-id=\"9052\" width=\"191\" data-init-width=\"200\" height=\"191\" data-init-height=\"200\" title=\"Deep Learning Courses\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2023\/02\/2-1.png\" data-width=\"191\" data-height=\"191\" style=\"--smush-placeholder-width: 191px; --smush-placeholder-aspect-ratio: 191\/191;aspect-ratio: auto 200 \/ 200;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Deep Learning Courses\" data-id=\"9052\" width=\"191\" data-init-width=\"200\" height=\"191\" data-init-height=\"200\" title=\"Deep Learning Courses\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2023\/02\/2-1.png\" data-width=\"191\" data-height=\"191\" style=\"aspect-ratio: auto 200 \/ 200;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Deep Learning Course<\/h4>\n<p style=\"\">Rating: <strong>4.5\/5<\/strong><\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"Natural Language Processing Course\" data-id=\"9053\" width=\"191\" data-init-width=\"200\" height=\"191\" data-init-height=\"200\" title=\"Natural Language Processing Course\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2023\/02\/3-1.png\" data-width=\"191\" data-height=\"191\" style=\"--smush-placeholder-width: 191px; --smush-placeholder-aspect-ratio: 191\/191;aspect-ratio: auto 200 \/ 200;\" mt-d=\"-19\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Natural Language Processing Course\" data-id=\"9053\" width=\"191\" data-init-width=\"200\" height=\"191\" data-init-height=\"200\" title=\"Natural Language Processing Course\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2023\/02\/3-1.png\" data-width=\"191\" data-height=\"191\" style=\"aspect-ratio: auto 200 \/ 200;\" mt-d=\"-19\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>NLP Course<\/h4>\n<p style=\"\">Rating: <strong>4.5\/5<\/strong><\/p>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>|\u00a0<span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span>\u00a0|\u00a0<span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span>\u00a0|\u00a0<span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0|\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions ? or want me to write an article on a specific topic? then feel free to comment below.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Cross validation defined as: \u201cA statistical method or a resampling procedure used to evaluate the skill of machine learning models on a limited data sample.\u201d It is mostly used while building machine learning models. It compares and selects a model for a given predictive modeling problem, assesses the models\u2019 predictive performance.&nbsp; Later judges how they<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/cross-validation\/\">+ Read More<\/a><\/p>\n","protected":false},"author":15,"featured_media":7615,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[6],"tags":[2107],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/12\/1-Cross-Validation.png","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/7611"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=7611"}],"version-history":[{"count":7,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/7611\/revisions"}],"predecessor-version":[{"id":11338,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/7611\/revisions\/11338"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media\/7615"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=7611"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=7611"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=7611"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}