{"id":1316,"date":"2017-01-09T09:30:22","date_gmt":"2017-01-09T09:30:22","guid":{"rendered":"http:\/\/dataaspirant.com\/?p=1316"},"modified":"2023-12-06T13:53:13","modified_gmt":"2023-12-06T08:23:13","slug":"knn-implementation-r-using-caret-package","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/knn-implementation-r-using-caret-package\/","title":{"rendered":"KNN R, K-Nearest Neighbor implementation in R using caret package"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><div id=\"attachment_1355\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1355\" class=\"wp-image-1355 size-full lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1.jpg\" alt=\"knn classifier implementation with r caret package\" width=\"1024\" height=\"768\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1.jpg 1024w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1-300x225.jpg 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1-768x576.jpg 768w\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/768;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1355\" class=\"wp-image-1355 size-full\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1.jpg\" alt=\"knn classifier implementation with r caret package\" width=\"1024\" height=\"768\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1.jpg 1024w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1-300x225.jpg 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1-768x576.jpg 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/noscript><p id=\"caption-attachment-1355\" class=\"wp-caption-text\">knn classifier implementation with r caret package<\/p><\/div>\n<h1 style=\"text-align: center;\">Knn classifier implementation in R with caret package<\/h1>\n<p style=\"text-align: justify;\">In this article, we are going to build a Knn classifier using R programming language. We will use the R <a href=\"https:\/\/dataaspirant.com\/category\/machine-learning-2\/\" target=\"_blank\">machine learning<\/a> caret package to build our Knn classifier. In our previous article,\u00a0we discussed the core concepts behind K-nearest neighbor algorithm. If you don&#8217;t have the basic understanding of Knn\u00a0algorithm, it&#8217;s suggested to read our\u00a0<a style=\"background-color: #f5f6f5;\" href=\"https:\/\/dataaspirant.com\/2016\/12\/23\/k-nearest-neighbor-classifier-intro\/\" target=\"_blank\">introduction to k-nearest neighbor<\/a>\u00a0article.<\/p>\n<p style=\"text-align: justify;\">For Knn classifier implementation in R programming language using caret package, we are going to examine a <b>wine dataset<\/b>. Our motive is to predict the origin of the wine. As in our <a href=\"https:\/\/dataaspirant.com\/2017\/01\/02\/k-nearest-neighbor-classifier-implementation-r-scratch\/\" target=\"_blank\">Knn\u00a0implementation in R <\/a>programming\u00a0post, we built a Knn classifier in R from scratch, but that process is not a feasible solution while working on big datasets.<\/p>\n<p style=\"text-align: justify;\"><!--more--><\/p>\n<p style=\"text-align: justify;\">To work on big datasets, we can directly use some machine learning packages. Developer community of R programming language has built some great packages to make our work easier. The beauty of these packages is that they are well optimized and can handle maximum\u00a0exceptions\u00a0to make our job simple, we just need to call functions for implementing algorithms with the\u00a0right parameters. For machine learning caret package is a nice package with proper documentation.<\/p>\n<p style=\"text-align: justify;\">The principle behind KNN classifier (K-Nearest Neighbor) algorithm is to find K predefined number of training samples that are <a href=\"https:\/\/dataaspirant.com\/2015\/04\/11\/five-most-popular-similarity-measures-implementation-in-python\/\" target=\"_blank\">closest in the distance<\/a> to a new point &amp; predict a label for our new point using these samples.<\/p>\n<h2 style=\"text-align: justify;\">Euclidean Distance<\/h2>\n<p style=\"text-align: justify;\">The most commonly used distance measure is\u00a0<a href=\"http:\/\/en.wikipedia.org\/wiki\/Euclidean_distance\" target=\"_blank\">Euclidean distance<\/a>. The Euclidean distance is also known as simply distance. The usage of Euclidean distance measure is highly recommended when data is dense or continuous. Euclidean distance is the best proximity measure.<\/p>\n<p style=\"text-align: justify;\">KNN classifier is also considered to be an instance based learning \/ non-generalizing algorithm. It stores records of training data in a multidimensional space. For each new sample &amp; particular value of K, it recalculates Euclidean distances and predicts the target class. So, it does not create\u00a0a generalized internal model.<\/p>\n<h2>Caret Package Installation<\/h2>\n<p style=\"text-align: justify;\">The R programming machine learning caret package( <span style=\"text-decoration: underline;\">C<\/span>lassification <span style=\"text-decoration: underline;\">A<\/span>nd <span style=\"text-decoration: underline;\">RE<\/span>gression <span style=\"text-decoration: underline;\">T<\/span>raining) holds tons of functions that helps to build predictive models. It holds tools for data splitting, pre-processing, feature selection, tuning and <a href=\"https:\/\/dataaspirant.com\/2014\/09\/19\/supervised-and-unsupervised-learning\/\" target=\"_blank\">supervised &#8211; unsupervised learning<\/a> algorithms, etc. It is similar to <a href=\"http:\/\/scikit-learn.org\/stable\/\" target=\"_blank\">sklearn library<\/a> in python.<\/p>\n<p style=\"text-align: justify;\">For using it, we first need to install it. Open R console and install it by typing:<\/p>\n<pre class=\"lang:default decode:true\">install.packages(\"caret\")<\/pre>\n<p style=\"text-align: justify;\">caret package provides us direct access to various functions for training our model with various <a href=\"https:\/\/dataaspirant.com\/2016\/09\/24\/classification-clustering-alogrithms\/\" target=\"_blank\">machine learning algorithms<\/a> like Knn, SVM, decision tree,<a href=\"https:\/\/dataaspirant.com\/2014\/10\/02\/linear-regression\/\" target=\"_blank\"> linear regression<\/a>, etc.<\/p>\n<h2>Knn implementation with\u00a0caret\u00a0package<\/h2>\n<h3>Wine Recognition Data Set Description<\/h3>\n<div id=\"attachment_1329\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1329\" class=\"size-full wp-image-1329 lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset.jpg\" alt=\"Wine recognition with knn in R \" width=\"1024\" height=\"768\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset.jpg 1024w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset-300x225.jpg 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset-768x576.jpg 768w\" data-sizes=\"(max-width: 1024px) 100vw, 1024px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 1024px; --smush-placeholder-aspect-ratio: 1024\/768;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1329\" class=\"size-full wp-image-1329\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset.jpg\" alt=\"Wine recognition with knn in R \" width=\"1024\" height=\"768\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset.jpg 1024w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset-300x225.jpg 300w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/wine-dataset-768x576.jpg 768w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/noscript><p id=\"caption-attachment-1329\" class=\"wp-caption-text\"><strong>Wine recognition with knn in R<\/strong><\/p><\/div>\n<p style=\"text-align: justify;\">For this experiment, wines were grown in the same region in Italy but derived from 3 different cultivars. The analysis determined the quantities of 13 constituents found in each of the three types of wines. We have a dataset with 13 attributes having continuous values and one attribute with class labels of wine origin.<\/p>\n<p>Using the wine dataset our task is to build a model to recognize the origin of the\u00a0wine. The original owners of this dataset are\u00a0Forina, M. et al. , PARVUS,\u00a0Institute of Pharmaceutical and Food Analysis and Technologies, Via Brigata Salerno, 16147 Genoa, Italy. This wine dataset is hosted as open data on <a href=\"https:\/\/archive.ics.uci.edu\/ml\/machine-learning-databases\/wine\/wine.data\" target=\"_blank\">UCI machine learning<\/a> repository.<\/p>\n<h4 style=\"text-align: left;\">The 13 Attributes of the dataset are:<\/h4>\n<ol>\n<li style=\"text-align: left;\">Alcohol<\/li>\n<li style=\"text-align: left;\">Malic acid<\/li>\n<li style=\"text-align: left;\">Ash<\/li>\n<li style=\"text-align: left;\">Alkalinity of ash<\/li>\n<li style=\"text-align: left;\">Magnesium<\/li>\n<li style=\"text-align: left;\">Total phenols<\/li>\n<li style=\"text-align: left;\">Flavanoids<\/li>\n<li style=\"text-align: left;\">Nonflavonoids phenols<\/li>\n<li style=\"text-align: left;\">Proanthocyanins<\/li>\n<li style=\"text-align: left;\">Color intensity<\/li>\n<li style=\"text-align: left;\">Hue<\/li>\n<li style=\"text-align: left;\">OD280\/OD315 of diluted wines<\/li>\n<li style=\"text-align: left;\">Proline<\/li>\n<\/ol>\n<p style=\"text-align: justify;\">The attribute with the\u00a0class label is at index 1. It consists of 3 values 1, 2 &amp; 3. These class labels are going to be predicted by our KNN model.<\/p>\n<h3>Wine Recognition Problem Statement:<\/h3>\n<p>To model a classifier for classifying the origin of the wine. The classifier should predict whether the\u00a0wine is from origin &#8220;1&#8221; or &#8220;2&#8221; or &#8220;3&#8221;.<\/p>\n<h2 style=\"text-align: center;\">Knn classifier implementation in R with Caret Package<\/h2>\n<h3>R caret Library:<\/h3>\n<p style=\"text-align: justify;\">For implementing Knn in r, we only need to import\u00a0caret package. As we mentioned above, it helps to perform various tasks to perform our machine learning work.<\/p>\n<pre class=\"lang:r decode:true\" title=\"Caret Library\">library(caret)<\/pre>\n<h3>Data Import:<\/h3>\n<p style=\"text-align: justify;\">We are using wine dataset from <a href=\"https:\/\/archive.ics.uci.edu\/ml\/machine-learning-databases\/wine\/wine.data\" target=\"_blank\">UCI repository<\/a>. For importing the data and manipulating it, we are going to use\u00a0data frames. First of all, we need to download the dataset.<\/p>\n<pre class=\"lang:r decode:true \" title=\"Download the Wine Dataset\">dataurl &lt;- \"https:\/\/archive.ics.uci.edu\/ml\/machine-learning-databases\/wine\/wine.data\"\r\ndownload.file(url = dataurl, destfile = \"wine.data\")\r\nwine_df &lt;- read.csv(\"wine.data\", header = FALSE)<\/pre>\n<p style=\"text-align: justify;\">In dataurl vector, we are\u00a0putting URL of our wine data. Using <strong>download.file()<\/strong> method, we can download the data file from URL. For downloading, URL of data and destination file name should be mentioned as the\u00a0parameters of <strong>download.file()<\/strong> method. Here, our destfile parameter is set with value &#8220;wine.data&#8221;.<\/p>\n<p style=\"text-align: justify;\">For importing data into an R data frame, we can use <strong>read.csv()<\/strong> method with parameters as a file name and whether our dataset consists o the 1st row with a header or not. If a header row exists then, the header should be set <strong>TRUE<\/strong> else header should \u00a0set\u00a0to<strong> FALSE<\/strong>.<\/p>\n<p>For checking the structure of data frame we can call the function <strong>str<\/strong> over wine_df:<\/p>\n<pre class=\"lang:r highlight:0 decode:true\" title=\"Wine Dataset\">&gt; str(wine_df)\r\n'data.frame':\t178 obs. of  14 variables:\r\n $ V1 : int  1 1 1 1 1 1 1 1 1 1 ...\r\n $ V2 : num  14.2 13.2 13.2 14.4 13.2 ...\r\n $ V3 : num  1.71 1.78 2.36 1.95 2.59 1.76 1.87 2.15 1.64 1.35 ...\r\n $ V4 : num  2.43 2.14 2.67 2.5 2.87 2.45 2.45 2.61 2.17 2.27 ...\r\n $ V5 : num  15.6 11.2 18.6 16.8 21 15.2 14.6 17.6 14 16 ...\r\n $ V6 : int  127 100 101 113 118 112 96 121 97 98 ...\r\n $ V7 : num  2.8 2.65 2.8 3.85 2.8 3.27 2.5 2.6 2.8 2.98 ...\r\n $ V8 : num  3.06 2.76 3.24 3.49 2.69 3.39 2.52 2.51 2.98 3.15 ...\r\n $ V9 : num  0.28 0.26 0.3 0.24 0.39 0.34 0.3 0.31 0.29 0.22 ...\r\n $ V10: num  2.29 1.28 2.81 2.18 1.82 1.97 1.98 1.25 1.98 1.85 ...\r\n $ V11: num  5.64 4.38 5.68 7.8 4.32 6.75 5.25 5.05 5.2 7.22 ...\r\n $ V12: num  1.04 1.05 1.03 0.86 1.04 1.05 1.02 1.06 1.08 1.01 ...\r\n $ V13: num  3.92 3.4 3.17 3.45 2.93 2.85 3.58 3.58 2.85 3.55 ...\r\n $ V14: int  1065 1050 1185 1480 735 1450 1290 1295 1045 1045 ...<\/pre>\n<p style=\"text-align: justify;\">It shows that our data consists of <strong>178<\/strong> observations and <strong>14<\/strong> columns. Value ranges of all\u00a0attributes from V2-V14 are varying, so we will have to standardize the data before training our classifier.<\/p>\n<h3>Data Slicing<\/h3>\n<p style=\"text-align: justify;\">Data slicing is a step to split data into train and test set. Training data set can be used specifically for our model building. Test dataset should not be mixed up while building model.\u00a0Even during standardization, we should not standardize our test set.<\/p>\n<pre class=\"lang:r decode:true\" title=\"Split the Wine Dataset in to train and test datasets\">set.seed(3033)\r\nintrain &lt;- createDataPartition(y = wine_df$V1, p= 0.7, list = FALSE)\r\ntraining &lt;- wine_df[intrain,]\r\ntesting &lt;- wine_df[-intrain,]<\/pre>\n<p style=\"text-align: justify;\">The<strong> set.seed()<\/strong> method is used to make our work replicable. As we want our\u00a0readers to learn concepts by coding these snippets. To make our answers replicable, we need to set a seed value. During partitioning of data, it splits randomly but if our readers will pass the\u00a0same value in the <strong>set.seed()<\/strong> method. Then we both will get identical results.<\/p>\n<p style=\"text-align: justify;\">The caret package provides a method\u00a0createDataPartition() for partitioning our data into train and test set. We are passing 3 parameters. The &#8220;y&#8221; parameter takes the\u00a0value of variable according to which data needs to be partitioned. In our case, target variable is at V1, so we are passing wine_df$V1 (wine data frame&#8217;s V1 column).<\/p>\n<p style=\"text-align: justify;\">The &#8220;p&#8221; parameter holds a decimal value in the range of 0-1. It&#8217;s to show that percentage of the\u00a0split. We are using p=0.7. It means that data split should be done in 70:30 ratio. The &#8220;list&#8221; parameter is for whether to return a list or matrix. We are passing FALSE for not returning a list. The createDataPartition() method is returning a matrix &#8220;intrain&#8221; with record&#8217;s indices.<\/p>\n<p style=\"text-align: justify;\">By passing values of intrain, we are splitting training data and testing data.<br \/>\nThe line\u00a0<span class=\"lang:default decode:true crayon-inline\">training &lt;- wine_df[intrain,]<\/span>\u00a0is for putting the data from data frame to training data. Remaining data is saved in the\u00a0testing data frame,\u00a0<span class=\"lang:default highlight:0 decode:true crayon-inline\">testing &lt;- wine_df[-intrain,]<\/span>.<\/p>\n<p style=\"text-align: justify;\">For checking the dimensions of our training data frame and testing data frame, we can use these:<\/p>\n<pre class=\"lang:r highlight:0 decode:true\" title=\"Train and Test dataset dimensions\">&gt; dim(training); dim(testing);\r\n[1] 125 14\r\n[1] 53 14\r\n<\/pre>\n<h3>Preprocessing &amp; Training<\/h3>\n<p style=\"text-align: justify;\">Preprocessing is all about\u00a0correcting the problems in data before building a machine learning model using that data. Problems can be of many types like missing values, attributes with a\u00a0different range, etc.<\/p>\n<p style=\"text-align: justify;\">To check whether our data contains missing values or not, we can use <strong>anyNA()<\/strong> method. Here, NA means Not Available.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\" title=\"Checking for NAN Values \">&gt; anyNA(wine_df)\r\n[1] FALSE<\/pre>\n<p style=\"text-align: justify;\">Since it&#8217;s returning<strong> FALSE<\/strong>, it means we don&#8217;t have any missing values.<\/p>\n<h4 style=\"text-align: justify;\">Wine Dataset summarized details<\/h4>\n<p style=\"text-align: justify;\">For checking the summarized details of our data, we can use <strong>summary()<\/strong> method. It will give us a basic idea about our dataset&#8217;s attributes range.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\" title=\"Dataset Basic Statistics\">&gt; summary(wine_df)\r\n       V1              V2              V3              V4              V5              V6        \r\n Min.   :1.000   Min.   :11.03   Min.   :0.740   Min.   :1.360   Min.   :10.60   Min.   : 70.00  \r\n 1st Qu.:1.000   1st Qu.:12.36   1st Qu.:1.603   1st Qu.:2.210   1st Qu.:17.20   1st Qu.: 88.00  \r\n Median :2.000   Median :13.05   Median :1.865   Median :2.360   Median :19.50   Median : 98.00  \r\n Mean   :1.938   Mean   :13.00   Mean   :2.336   Mean   :2.367   Mean   :19.49   Mean   : 99.74  \r\n 3rd Qu.:3.000   3rd Qu.:13.68   3rd Qu.:3.083   3rd Qu.:2.558   3rd Qu.:21.50   3rd Qu.:107.00  \r\n Max.   :3.000   Max.   :14.83   Max.   :5.800   Max.   :3.230   Max.   :30.00   Max.   :162.00  \r\n       V7              V8              V9              V10             V11              V12        \r\n Min.   :0.980   Min.   :0.340   Min.   :0.1300   Min.   :0.410   Min.   : 1.280   Min.   :0.4800  \r\n 1st Qu.:1.742   1st Qu.:1.205   1st Qu.:0.2700   1st Qu.:1.250   1st Qu.: 3.220   1st Qu.:0.7825  \r\n Median :2.355   Median :2.135   Median :0.3400   Median :1.555   Median : 4.690   Median :0.9650  \r\n Mean   :2.295   Mean   :2.029   Mean   :0.3619   Mean   :1.591   Mean   : 5.058   Mean   :0.9574  \r\n 3rd Qu.:2.800   3rd Qu.:2.875   3rd Qu.:0.4375   3rd Qu.:1.950   3rd Qu.: 6.200   3rd Qu.:1.1200  \r\n Max.   :3.880   Max.   :5.080   Max.   :0.6600   Max.   :3.580   Max.   :13.000   Max.   :1.7100  \r\n      V13             V14        \r\n Min.   :1.270   Min.   : 278.0  \r\n 1st Qu.:1.938   1st Qu.: 500.5  \r\n Median :2.780   Median : 673.5  \r\n Mean   :2.612   Mean   : 746.9  \r\n 3rd Qu.:3.170   3rd Qu.: 985.0  \r\n Max.   :4.000   Max.   :1680.0<\/pre>\n<p style=\"text-align: justify;\">From above summary <a href=\"https:\/\/dataaspirant.com\/statistics-guide\/\" target=\"_blank\">statistics<\/a>, it shows us that all the attributes have a\u00a0different range. So, we need to standardize our data. We can standardize data using caret&#8217;s <strong>preProcess()<\/strong> method.<\/p>\n<p style=\"text-align: justify;\">Our target variable consists of 3 values 1, 2, 3. These should considered as categorical variables. To convert these to categorical variables, we can convert them to factors.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\">training[[\"V1\"]] = factor(training[[\"V1\"]])<\/pre>\n<p style=\"text-align: justify;\">The above line of code will convert training data frame&#8217;s &#8220;V1&#8221; column to factor variable.<\/p>\n<p style=\"text-align: justify;\">Now, it&#8217;s time to train our model.<\/p>\n<h3 style=\"text-align: justify;\">Training the Knn model<\/h3>\n<p style=\"text-align: justify;\">Caret package provides <strong>train()<\/strong> method for training our data for various algorithms. We just need to pass different parameter values for different algorithms. Before <strong>train()<\/strong> method, we will first\u00a0use <strong>trainControl()<\/strong> method. It controls the computational nuances of the\u00a0<strong>train()<\/strong> method.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\">trctrl &lt;- trainControl(method = \"repeatedcv\", number = 10, repeats = 3)\r\nset.seed(3333)\r\nknn_fit &lt;- train(V1 ~., data = training, method = \"knn\",\r\n trControl=trctrl,\r\n preProcess = c(\"center\", \"scale\"),\r\n tuneLength = 10)<\/pre>\n<p style=\"text-align: justify;\">We are setting 3 parameters of <strong>trainControl()<\/strong> method. The &#8220;method&#8221; parameter holds the details about resampling method. We can set &#8220;method&#8221; with many values like\u00a0\u00a0&#8220;boot&#8221;, &#8220;boot632&#8221;, &#8220;cv&#8221;, &#8220;repeatedcv&#8221;, &#8220;LOOCV&#8221;, &#8220;LGOCV&#8221; etc. For this tutorial, let&#8217;s try to use repeatedcv i.e, repeated cross-validation.<\/p>\n<p style=\"text-align: justify;\">The &#8220;number&#8221; parameter holds the number of resampling iterations. The &#8220;repeats &#8221; parameter contains the complete sets of folds to compute for our repeated cross-validation. We are using setting number =10 and repeats =3. This <strong>trainControl()<\/strong> methods returns a list. We are going to pass this on our <strong>train()<\/strong> method.<br \/>\nBefore training our knn classifier, <strong>set.seed()<\/strong>.<\/p>\n<p style=\"text-align: justify;\">For training knn classifier, <strong>train()<\/strong> method should be passed with &#8220;method&#8221; parameter as &#8220;knn&#8221;. We are passing our target variable V1. The V1~. denotes a formula for using all attributes in our classifier and V1 as the target variable. The &#8220;trControl&#8221; parameter should be passed with results from our <strong>trianControl()<\/strong> method. The &#8220;preProcess&#8221; \u00a0parameter is for preprocessing our training data.<\/p>\n<p style=\"text-align: justify;\">As discussed earlier for our data, preprocessing is a mandatory task. We are passing 2 values in our &#8220;preProcess&#8221; parameter &#8220;center&#8221; &amp; &#8220;scale&#8221;. These two help for centering and scaling the data. After preProcessing these convert our training data with mean value as approximately &#8220;0&#8221; and standard deviation as &#8220;1&#8221;. The &#8220;tuneLength&#8221; parameter holds an integer value.\u00a0This is for tuning our algorithm.<\/p>\n<h4 style=\"text-align: justify;\">Trained Knn model result<\/h4>\n<p style=\"text-align: justify;\">You can check the result of our <strong>train()<\/strong> method. We are saving its results in a knn_fit variable.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\">&gt; knn_fit\r\nk-Nearest Neighbors \r\n\r\n125 samples\r\n 13 predictor\r\n 3 classes: '1', '2', '3' \r\n\r\nPre-processing: centered (13), scaled (13) \r\nResampling: Cross-Validated (10 fold, repeated 3 times) \r\nSummary of sample sizes: 114, 112, 113, 112, 114, 112, ... \r\nResampling results across tuning parameters:\r\n\r\n k Accuracy Kappa \r\n 5 0.9543790 0.9317929\r\n 7 0.9404512 0.9109657\r\n 9 0.9379260 0.9073292\r\n 11 0.9374598 0.9067419\r\n 13 0.9396270 0.9099077\r\n 15 0.9482129 0.9225977\r\n 17 0.9479604 0.9222815\r\n 19 0.9516706 0.9276711\r\n 21 0.9597597 0.9401666\r\n 23 0.9432678 0.9152521\r\n\r\nAccuracy was used to select the optimal model using the largest value.\r\nThe final value used for the model was k = 21.<\/pre>\n<p style=\"text-align: justify;\">Its showing Accuracy and Kappa metrics result for different k value. From the results, it automatically selects best k-value. Here, our training model is choosing k = 21 as its final value.<\/p>\n<p style=\"text-align: justify;\">We can see variation in Accuracy w.r.t K value by plotting these in a graph.<\/p>\n<div><\/div>\n<div>\n<div id=\"attachment_1361\" style=\"width: 650px\" class=\"wp-caption aligncenter\"><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1361\" class=\"wp-image-1361 size-full lazyload\" data-src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Rplot01.png\" alt=\"Accuracy vs K-Value\" width=\"640\" height=\"358\" data-srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Rplot01.png 640w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Rplot01-300x168.png 300w\" data-sizes=\"(max-width: 640px) 100vw, 640px\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" style=\"--smush-placeholder-width: 640px; --smush-placeholder-aspect-ratio: 640\/358;\" \/><noscript><img loading=\"lazy\" decoding=\"async\" aria-describedby=\"caption-attachment-1361\" class=\"wp-image-1361 size-full\" src=\"http:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Rplot01.png\" alt=\"Accuracy vs K-Value\" width=\"640\" height=\"358\" srcset=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Rplot01.png 640w, https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Rplot01-300x168.png 300w\" sizes=\"(max-width: 640px) 100vw, 640px\" \/><\/noscript><p id=\"caption-attachment-1361\" class=\"wp-caption-text\">Accuracy vs K-Value<\/p><\/div>\n<\/div>\n<h3>Test Set Prediction<\/h3>\n<p style=\"text-align: justify;\">Now, our model is trained with K value as 21. We are ready to predict classes for our test set. We can use predict() method.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\">&gt; test_pred &lt;- predict(knn_fit, newdata = testing)\r\n&gt; test_pred\r\n [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 2 2 2 2 2 2 2 3 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2\r\n[40] 3 3 3 3 3 3 3 3 3 3 3 3 3 3\r\nLevels: 1 2 3<\/pre>\n<p style=\"text-align: justify;\">caret package provides predict() method for predicting results. We are passing 2 arguments. The first parameter is our trained model and second parameter &#8220;newdata&#8221; holds our testing data frame. The predict() method returns a list, we are saving it in a test_pred variable.<\/p>\n<h4 style=\"text-align: justify;\">How Accurately our model is working?<\/h4>\n<p style=\"text-align: justify;\">Using confusion matrix, we can print statistics of our results. It shows that our model\u00a0accuracy for test set is 96.23%.<\/p>\n<pre class=\"lang:r highlight:0 decode:true\">&gt; confusionMatrix(test_pred, testing$V1 )\r\nConfusion Matrix and Statistics\r\n\r\n Reference\r\nPrediction 1 2 3\r\n 1 15 0 0\r\n 2 0 22 0\r\n 3 0 2 14\r\n\r\nOverall Statistics\r\n \r\n Accuracy : 0.9623 \r\n 95% CI : (0.8702, 0.9954)\r\n No Information Rate : 0.4528 \r\n P-Value [Acc &gt; NIR] : 1.208e-15 \r\n \r\n Kappa : 0.9421 \r\n Mcnemar's Test P-Value : NA \r\n\r\nStatistics by Class:\r\n\r\n Class: 1 Class: 2 Class: 3\r\nSensitivity 1.000 0.9167 1.0000\r\nSpecificity 1.000 1.0000 0.9487\r\nPos Pred Value 1.000 1.0000 0.8750\r\nNeg Pred Value 1.000 0.9355 1.0000\r\nPrevalence 0.283 0.4528 0.2642\r\nDetection Rate 0.283 0.4151 0.2642\r\nDetection Prevalence 0.283 0.4151 0.3019\r\nBalanced Accuracy 1.000 0.9583 0.9744\r\n<\/pre>\n<p>You can download code from\u00a0our github code repo. It can be accessed from<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/tree\/master\/K-nearest-neighbor%20classifier\/Knn-Classifier-R-caret\" target=\"_blank\" rel=\"noopener\"> here.<\/a><\/p>\n<h3>Related Articles To Read<\/h3>\n<ul>\n<li><a href=\"https:\/\/dataaspirant.com\/2016\/12\/23\/k-nearest-neighbor-classifier-intro\/\" target=\"_blank\" rel=\"noopener\">Introduction to knn algorithm<\/a><\/li>\n<li><a href=\"https:\/\/dataaspirant.com\/2016\/12\/27\/k-nearest-neighbor-algorithm-implementaion-python-scratch\/\" target=\"_blank\" rel=\"noopener\">Knn algorithm implementation purely in python without any machine learning libraries<\/a><\/li>\n<li><a href=\"https:\/\/dataaspirant.com\/2016\/12\/30\/k-nearest-neighbor-implementation-scikit-learn\/\" target=\"_blank\" rel=\"noopener\">Cancer tumor detection with knn sklearn<\/a><\/li>\n<li><a href=\"https:\/\/dataaspirant.com\/2017\/01\/02\/k-nearest-neighbor-classifier-implementation-r-scratch\/\" target=\"_blank\" rel=\"noopener\">Knn implementation in R without any libraries<\/a><\/li>\n<\/ul>\n<h3>References:<\/h3>\n<ul>\n<li><a href=\"https:\/\/archive.ics.uci.edu\/ml\/datasets\/Wine\" target=\"_blank\" rel=\"noopener\">Wine Dataset<\/a>, Lichman, M. (2013). <a href=\"http:\/\/archive.ics.uci.edu\/ml\" target=\"_blank\" rel=\"noopener\">UCI Machine Learning Repository<\/a>.<\/li>\n<li>caret package <a href=\"http:\/\/topepo.github.io\/caret\/index.html\" target=\"_blank\" rel=\"noopener\">documentation<\/a><\/li>\n<\/ul>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\" rel=\"noopener\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>| <span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span> | <span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>| <span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span> | <span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0| <a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions, then feel free to comment below.\u00a0 If you want me to write on one particular topic, then do tell it to me in the comments below.<\/p>\n<h3>Related Courses:<\/h3>\n<p>Do check out <a href=\"https:\/\/dataaspirant.com\/data-science-courses\/\" target=\"_blank\">unlimited data science courses<\/a><\/p>\n<table style=\"height: 146px;\" width=\"714\">\n<tbody>\n<tr>\n<td style=\"width: 230.671875px;\"><strong>Title &amp; links<\/strong><\/td>\n<td style=\"width: 230.671875px;\"><strong>Details<\/strong><\/td>\n<td style=\"width: 230.671875px;\"><strong>What You Will Learn<\/strong><\/td>\n<\/tr>\n<tr>\n<td style=\"width: 230.671875px;\"><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.950390&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fmachinelearning%2F\" target=\"new\" rel=\"noopener\" target=\"_blank\">Machine Learning A-Z: Hands-On Python &amp; R In Data Science<\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.950390&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/p>\n<p><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.950390&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fmachinelearning%2F\" target=\"new\" rel=\"noopener\" target=\"_blank\"><img decoding=\"async\" data-src=\"https:\/\/udemy-images.udemy.com\/course\/480x270\/950390_270f_3.jpg\" border=\"0\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" \/><noscript><img decoding=\"async\" src=\"https:\/\/udemy-images.udemy.com\/course\/480x270\/950390_270f_3.jpg\" border=\"0\" \/><\/noscript><\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.950390&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/td>\n<td style=\"width: 230.671875px;\"><span class=\"rating-and-enrolled__element\">Students Enrolled ::\u00a0<strong>19,359<\/strong><\/span><\/p>\n<p>Course Overall Rating::\u00a0<strong>4.6\u00a0<\/strong><\/td>\n<td style=\"width: 230.671875px;\">\n<ul class=\"regular-list\">\n<li>Master Machine Learning on Python &amp; R<\/li>\n<li>Make robust Machine Learning models.<\/li>\n<li>Handle specific topics like Reinforcement Learning, NLP and Deep Learning.<\/li>\n<li>Build an army of powerful Machine Learning models and know how to combine them to solve any problem.<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 230.671875px;\"><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.765242&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fr-programming%2F\" target=\"new\" rel=\"noopener\" target=\"_blank\">R Programming A-Z: R For Data Science With Real Exercises!<\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.765242&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/p>\n<p><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.765242&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fr-programming%2F\" target=\"new\" rel=\"noopener\" target=\"_blank\"><img decoding=\"async\" data-src=\"https:\/\/udemy-images.udemy.com\/course\/480x270\/765242_2578.jpg\" border=\"0\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" \/><noscript><img decoding=\"async\" src=\"https:\/\/udemy-images.udemy.com\/course\/480x270\/765242_2578.jpg\" border=\"0\" \/><\/noscript><\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.765242&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><\/td>\n<td style=\"width: 230.671875px;\"><span class=\"rating-and-enrolled__element\">Students Enrolled :: <strong>12,001<\/strong><\/span><\/p>\n<p>Course Overall Rating ::\u00a0<strong>4.6<\/strong><\/td>\n<td style=\"width: 230.671875px;\">\n<ul>\n<li>Program in R at a good level.<\/li>\n<li>Learn the core principles of programming.<\/li>\n<li>Understand the Normal distribution.<\/li>\n<li>Practice working with statistical, financial and sport data in R<\/li>\n<\/ul>\n<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 230.671875px;\"><a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.550998&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fdata-mining-with-r-go-from-beginner-to-advanced%2F\" target=\"new\" rel=\"noopener\" target=\"_blank\">Data Mining with R: Go from Beginner to Advanced!<\/a><img loading=\"lazy\" decoding=\"async\" src=\"http:\/\/ad.linksynergy.com\/fs-bin\/show?id=d1QCig2q3qI&amp;bids=323058.550998&amp;type=2&amp;subid=0\" width=\"1\" height=\"1\" border=\"0\" \/><br \/>\n<a href=\"http:\/\/click.linksynergy.com\/link?id=d1QCig2q3qI&amp;offerid=323058.550998&amp;type=2&amp;murl=https%3A%2F%2Fwww.udemy.com%2Fdata-mining-with-r-go-from-beginner-to-advanced%2F\" target=\"new\" rel=\"noopener\" target=\"_blank\"><img decoding=\"async\" data-src=\"https:\/\/udemy-images.udemy.com\/course\/480x270\/550998_db4f_2.jpg\" border=\"0\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\" \/><noscript><img decoding=\"async\" src=\"https:\/\/udemy-images.udemy.com\/course\/480x270\/550998_db4f_2.jpg\" border=\"0\" \/><\/noscript><\/a><\/td>\n<td style=\"width: 230.671875px;\"><span class=\"rating-and-enrolled__element\">Students Enrolled :: <strong>2,380<\/strong><\/span><\/p>\n<p>Course Overall\u00a0Rating ::\u00a0<strong>4.2<\/strong><\/td>\n<td style=\"width: 230.671875px;\">\n<ul>\n<li>Use R software for data import and export, data exploration and visualization, and for data analysis tasks, including performing a comprehensive set of data mining operations.<\/li>\n<li>Apply the dozens of included &#8220;hands-on&#8221; cases and examples using real data and R scripts to new and unique data analysis and data mining problems.<\/li>\n<\/ul>\n<ul>\n<li class=\"what-you-get__item\"><span class=\"what-you-get__text\">Effectively use a number of popular, contemporary data mining methods and techniques in demand by industry\u00a0including:\u00a0(1) Decision, classification and regression trees (CART); (2) Random forests; (3) Linear and logistic regression; and (4) Various cluster analysis techniques.<\/span><\/li>\n<\/ul>\n<div class=\"what-you-get__item\"><\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><span class=\"rating-and-enrolled__element\" style=\"min-width: 1px; margin-right: 10px; color: #353535; font-family: 'Open Sans', 'Helvetica Neue', Helvetica, Arial, sans-serif; font-size: 15px;\">\u00a0<\/span><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Knn classifier implementation in R with caret package In this article, we are going to build a Knn classifier using R programming language. We will use the R machine learning caret package to build our Knn classifier. In our previous article,\u00a0we discussed the core concepts behind K-nearest neighbor algorithm. If you don&#8217;t have the basic<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/knn-implementation-r-using-caret-package\/\">+ Read More<\/a><\/p>\n","protected":false},"author":5,"featured_media":1355,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[6,9],"tags":[13,39,15,18,51,21,52,33],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/dataaspirant.com\/wp-content\/uploads\/2017\/01\/Knn-implementation-R-caret-1.jpg","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1316"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=1316"}],"version-history":[{"count":1,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1316\/revisions"}],"predecessor-version":[{"id":11855,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/1316\/revisions\/11855"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media\/1355"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=1316"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=1316"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=1316"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}