{"id":7515,"date":"2020-11-30T02:55:14","date_gmt":"2020-11-30T02:55:14","guid":{"rendered":"https:\/\/dataaspirant.com\/?p=7515"},"modified":"2023-12-06T13:53:53","modified_gmt":"2023-12-06T08:23:53","slug":"tf-idf-term-frequency-inverse-document-frequency","status":"publish","type":"post","link":"https:\/\/dataaspirant.com\/tf-idf-term-frequency-inverse-document-frequency\/","title":{"rendered":"How TF-IDF, Term Frequency-Inverse Document Frequency Works"},"content":{"rendered":"<div id=\"bsf_rt_marker\"><\/div><p><span><img decoding=\"async\" alt=\"How TF-IDF Works\" data-id=\"7517\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"How TF-IDF works\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/1-How-TF-IDF-works.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;aspect-ratio: auto 750 \/ 450;\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"How TF-IDF Works\" data-id=\"7517\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"How TF-IDF works\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/1-How-TF-IDF-works.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"aspect-ratio: auto 750 \/ 450;\"><\/noscript><\/span><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For building any natural language model, the key challenge is how to convert the <strong>text data into numerical data<\/strong>. As the machine learning or deep learning models don\u2019t understand the text data. One smart way to do the conversion is the <strong>TF-IDF<\/strong> method.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This TF-IDF method is a popular word embedding technique used in various natural language processing tasks. We have <a href=\"https:\/\/dataaspirant.com\/word-embedding-techniques-nlp\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>different other word embedding techniques<\/strong><\/a> to convert the text data to numerical data.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">But In this article, we talk about TF-IDF. Which is in short for \u201c<strong>term frequency\u2013inverse document frequency<\/strong>.\u201d<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This numerical <a href=\"https:\/\/dataaspirant.com\/statistics-guide\/\" target=\"_blank\">statistical<\/a> technique is used to figure out the relevance of any word in the document, which is a part of an even larger body of documents.<\/p>\n<p dir=\"ltr\">Evident from the name itself. It is composed of<strong> two<\/strong> different terms:&nbsp;<\/p>\n<ul>\n<li dir=\"ltr\"><strong>TF:<\/strong> Measures how many times a <strong>word appears<\/strong> in the document.<\/li>\n<li dir=\"ltr\"><strong>IDF:<\/strong> Represents how <strong>common<\/strong> the word is across the different documents.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">This technique has many use-cases.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For example, TF-IDF is very popular for scoring the words in machine learning algorithms that work with textual data (for example, Natural Language Processing tasks like <a href=\"https:\/\/dataaspirant.com\/build-email-spam-classification-model-spacy-python\/\" target=\"_blank\" rel=\"noopener\"><strong>Email spam detection<\/strong><\/a>).<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">TF-IDF is considered as a weighting factor in tasks like information retrieval and <a href=\"https:\/\/dataaspirant.com\/data-mining\/\" target=\"_blank\" rel=\"noopener\"><strong>data mining<\/strong><\/a>. We will talk about it in more detail in the coming sections.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The target audience for this article is <strong>not limited<\/strong> to just machine learning practitioners or researchers. But also people from domains like <strong>search engine optimization<\/strong> who wish to work with TF-IDF to improve their site\/blog rankings.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Unlike the name suggests. We will see how this algorithm is not complicated at all.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The reader is not expected to have any prerequisite knowledge of the topic. But later in the section where the topic is discussed with <strong>code implementation<\/strong>, basic Python language knowledge can come in handy.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">With this being said, let&#8217;s start the article. Below are the topics you are going to learn in this article, only if you read the complete article \ud83d\ude42<\/p>\n<h2 id=\"t-1606649429002\">Common NLP Terminologies<\/h2>\n<p><span><img decoding=\"async\" alt=\"Common NLP Terminologies\" data-id=\"7527\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Common NLP Terminologies\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/2-Common-NLP-Terminologies.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;aspect-ratio: auto 750 \/ 450;\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Common NLP Terminologies\" data-id=\"7527\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"Common NLP Terminologies\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/2-Common-NLP-Terminologies.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"aspect-ratio: auto 750 \/ 450;\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Common NLP Terminologies<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Before going further deep into the topic, it is imperative for us to understand the fundamental terminologies used in this context.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If these fundamental concepts are clear, it will be effortless to understand how the <strong>frequency\u2013inverse document frequency<\/strong> converts the text data to the numerical data.<\/p>\n<h3 dir=\"ltr\" id=\"t-1606649429003\">What is Corpus?<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">For different people, the term corpus can have various meanings, but we are interested in what it means to <strong>computational linguists<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">A corpus (from the Latin word Corpora) can be thought of as a representative sample of the problem and a collection of text data. When this corpus is structured and annotated, it is more commonly known as a <strong>labeled corpus<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In the <a href=\"https:\/\/dataaspirant.com\/category\/machine-learning-2\/\" target=\"_blank\" rel=\"noopener\"><strong>machine learning context<\/strong><\/a>, it refers to an extensive collection of text data from sources like books, news, webpages, etc., which forms the basis for various linguistic analysis tasks.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\"><strong>In short:<\/strong> Corpus is a collection of words, we are using for testing our hypothesis or for building models.<\/p>\n<h3 dir=\"ltr\" id=\"t-1606649429004\">What is Term Frequency?<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">At heart, it is a measure of how frequently a term (word) occurs in a document. A simple way to calculate it is just to have a <strong>raw count<\/strong> of the number of times a certain term appeared in the document.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">But there is a need to make some adjustments to not receive biased results. This is due to the fact that different documents can be of varying lengths, and it is possible that a term will appear more often in a longer document than in a shorter one.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">To compensate for this ballgame, this frequency value needs to be <strong>normalized<\/strong>. This is done by dividing the raw count of the term in the document by the length of the document or by the count of the most common (frequent) term in that document.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">More intuitively, it can be accounted like this:<\/p>\n<p dir=\"ltr\" style=\"text-align: center;\"><strong>TF(\u2018xyz\u2019) = (Number of times term \u2018xyz\u2019 appeared in the document) \/ (Length of the document).<\/strong><\/p>\n<h3 dir=\"ltr\" id=\"t-1606649429005\">What is Inverse Document Frequency?<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">This measure tells how rare (or common) a term is in the entire set of documents (popularly known as a Corpus).&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Intuitively, the more common a term is, the less important it will be. In this case the value of the IDF will be <strong>closer to 0<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Terms like <strong>\u2018is,\u2019 \u2018are,<\/strong>\u2019 and <strong>\u2018of,<\/strong>\u2019 they appear very often but have very little importance, which <strong>doesn\u2019t get<\/strong> reflected properly in the TF measure.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">However, for a rare word, it will be precisely contrary to the former. Such terms should be given more attention while measuring their relevance in the set of documents.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This measure is calculated by <strong>dividing<\/strong> the total number of documents by the number of documents containing that word, and then taking a logarithm, or more intuitively by estimating the following:<\/p>\n<p dir=\"ltr\" style=\"text-align: center;\"><strong>IDF(\u2018xyz\u2019) = log_e (Number of documents in corpus \/ Number of documents with term \u2018xyz\u2019 in it)<\/strong><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Thus, to weigh down the frequent terms and scale up the not-so-common terms. An IDF factor needs to be incorporated to distinguish between the <strong>relevant vs. non-relevant terms<\/strong> as well as documents.&nbsp;<\/p>\n<h3 dir=\"ltr\" id=\"t-1606649429006\">TF-IDF ( Term Frequency &#8211; Inverse Document Frequency )<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">Subsequently, the TF-IDF value can be calculated by taking a product of the two statistics: TF and IDF. Intuitively, this means:<\/p>\n<p dir=\"ltr\" style=\"text-align: center;\"><strong>TF-IDF (\u2018xyz\u2019) = TF(\u2018xyz\u2019) * IDF(\u2018xyz\u2019)<\/strong><\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">A term will be considered highly relevant in a document and assigned a higher weight when it has a <strong>high TF<\/strong> value (in that document) and a low document frequency in the whole corpus.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The IDF value in that situation will be greater than 1 (since the log value is greater than 1), and thus the overall TF-IDF value will be large.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">However, when a term is more common in the corpus, the TF-IDF value drops down <strong>closer to 0<\/strong>.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\"><strong>Note:<\/strong> While we only discussed calculating the TF and IDF values in the simplest way, there exists a number of different ways to calculate their exact value. Which will be beyond this introductory article.&nbsp;<\/p>\n<h2 dir=\"ltr\" id=\"t-1606649429007\">TF-IDF Calculation<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">We learned everything we need to compute the TF-IDF. Now Let\u2019s take a toy kind of example to perform the calculation.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">First, let\u2019s see how we can calculate the TF-IDF value using the <strong>excel<\/strong>. Next we will see how we can calculate with <strong>simple python<\/strong> code.<\/p>\n<h3 dir=\"ltr\" id=\"t-1606649429008\" style=\"text-align: justify;\">TF-IDF Calculation in Excel<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">For this purpose, we are going to take <strong>2<\/strong> documents. Each contains <strong>1<\/strong> sentence. Considering this as input text, we will calculate the TF-IDF value.<\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"TF-IDF Calculation Example\" data-id=\"7530\" width=\"638\" data-init-width=\"1024\" height=\"171\" data-init-height=\"274\" title=\"TF-IDF Calculation Example\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/3-TF-IDF-Calculation-Example-1024x274.png\" data-width=\"638\" data-height=\"171\" style=\"--smush-placeholder-width: 638px; --smush-placeholder-aspect-ratio: 638\/171;aspect-ratio: auto 1024 \/ 274;\" ml-d=\"-12\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"TF-IDF Calculation Example\" data-id=\"7530\" width=\"638\" data-init-width=\"1024\" height=\"171\" data-init-height=\"274\" title=\"TF-IDF Calculation Example\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/3-TF-IDF-Calculation-Example-1024x274.png\" data-width=\"638\" data-height=\"171\" style=\"aspect-ratio: auto 1024 \/ 274;\" ml-d=\"-12\" loading=\"lazy\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">TF-IDF Calculation Example<\/p>\n<p dir=\"ltr\">Here we are considering two documents, each containing the below sentences.<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\"><strong>Document 1: <\/strong>I read the SVM algorithm article in dataaspirant blog<\/li>\n<li dir=\"ltr\"><strong>Document 2:<\/strong> I read the randomforest algorithm article in dataaspirant blog<\/li>\n<\/ul>\n<p dir=\"ltr\">After the <a href=\"https:\/\/dataaspirant.com\/nlp-text-preprocessing-techniques-implementation-python\/\" target=\"_blank\" rel=\"noopener\"><strong>text preprocessing<\/strong><\/a> step, we will end up with the below sentences.<\/p>\n<ul>\n<li ltr\"\"=\"\"><strong>Document<\/strong> 1: read SVM algorithm article dataaspirant blog<\/li>\n<li><strong>Document<\/strong> 2: read randomforest algorithm article dataaspirant blog<\/li>\n<\/ul>\n<p><span><img decoding=\"async\" alt=\"Term Frequency and Inverse Document Frequency Calculation\" data-id=\"7533\" width=\"626\" data-init-width=\"2252\" height=\"160\" data-init-height=\"574\" title=\"Term Frequency and Inverse Document Frequency Calculation\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/4-Term-Frequency-and-Inverse-Document-Frequency-Calculation.png\" data-width=\"626\" data-height=\"160\" loading=\"lazy\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/160;aspect-ratio: auto 2252 \/ 574;\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Term Frequency and Inverse Document Frequency Calculation\" data-id=\"7533\" width=\"626\" data-init-width=\"2252\" height=\"160\" data-init-height=\"574\" title=\"Term Frequency and Inverse Document Frequency Calculation\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/4-Term-Frequency-and-Inverse-Document-Frequency-Calculation.png\" data-width=\"626\" data-height=\"160\" loading=\"lazy\" style=\"aspect-ratio: auto 2252 \/ 574;\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">Term Frequency and Inverse Document Frequency Calculation<\/p>\n<ul>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">After preprocessing, we list down all the unique words for performing the TF-IDF calculation.<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">As a first step, we count the number of times the word came in the documents.<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">For example, for the word <strong>read<\/strong> appeared once in <strong>document-1<\/strong> and once in the <strong>document-2<\/strong>.<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">In the second step, we calculated the TF (term frequency)<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">For example, for the word read, TF is<strong> 0.17<\/strong>, which is 1 (word count) \/ 6 (number of words in document-1)<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">In the third step, we calculated the IDF inverse document frequency.<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">For example for the word read IDF is 0, which is <strong>log<\/strong>(2 (number of documents) \/ 2 (In number of documents word read present))&nbsp;<\/li>\n<li ltr\"\"=\"\" style=\"text-align: justify;\">In the fourth step, we calculated the TF * IDF.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">If you see clearly, the key difference between these two documents is the <strong>algorithm<\/strong> they are talking about.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For the first document, it is about the <a href=\"https:\/\/dataaspirant.com\/support-vector-machine-algorithm\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>SVM algorithm<\/strong><\/a>, whereas for the second document, it is about <a href=\"https:\/\/dataaspirant.com\/random-forest-algorithm-machine-learing\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>randomforest algorithm<\/strong><\/a>.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This is clearly coming up in the <strong>TF-IDF<\/strong> calculation. For all other words, we are getting <strong>0<\/strong>, and for <strong>SVM and randomforest<\/strong> words, we are getting some value.<\/p>\n<h3 dir=\"ltr\" id=\"t-1606649429009\">TF-IDF Calculation With Simple Python Code<\/h3>\n<p dir=\"ltr\" style=\"text-align: justify;\">For this purpose, we are going to take <strong>3 documents<\/strong>. Each contains <strong>4 sentences<\/strong>. Considering this as input text, we will calculate the TF-IDF value.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">You can <strong>scroll<\/strong> the above notebook to check each step you need to calculate the TF-IDF value.&nbsp;<\/p>\n<h2 dir=\"ltr\" id=\"t-1606649429010\">Using TF-IDF in Machine Learning Models<\/h2>\n<p><span><img decoding=\"async\" alt=\"TF-IDF In Machine Learning\" data-id=\"7540\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"TF-IDF In Machine Learning\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/5-TF-IDF-In-Machine-Learning.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;aspect-ratio: auto 750 \/ 450;\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"TF-IDF In Machine Learning\" data-id=\"7540\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"TF-IDF In Machine Learning\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/5-TF-IDF-In-Machine-Learning.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"aspect-ratio: auto 750 \/ 450;\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">TF-IDF In Machine Learning<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">In a very general sense, <a href=\"https:\/\/dataaspirant.com\/decision-tree-algorithm-python-with-scikit-learn\/\" target=\"_blank\" rel=\"noopener\"><strong>machine learning models<\/strong><\/a> are very powerful and always ready to be fed some data to bring out some useful and sensible outcomes.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">However, a machine learning model is only as good as the type and quality of data it sees during training (and at other stages). It is, thus, our duty to make sure that the data is clean, well processed, and in a format which the model can <strong>understand<\/strong> easily.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The biggest hurdle with <a href=\"https:\/\/dataaspirant.com\/category\/natural-language-processing\/\" target=\"_blank\" rel=\"noopener\"><strong>Natural language processing<\/strong><\/a> (NLP) is that the algorithms deal with numbers, while the data we have in NLP is text.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">So, after performing the <a href=\"https:\/\/dataaspirant.com\/nlp-text-preprocessing-techniques-implementation-python\/\" target=\"_blank\" rel=\"noopener\"><strong>various text preprocessing<\/strong><\/a> steps like&nbsp;<\/p>\n<ul>\n<li dir=\"ltr\">Tokenization,&nbsp;<\/li>\n<li dir=\"ltr\">Stemming,<\/li>\n<li dir=\"ltr\">Lemmatization,<\/li>\n<li dir=\"ltr\">Stop-words removal etc.,<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">It is important that we convert the resulting <strong>text data into numbers<\/strong> (feature vector), which is understood by the model.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For this we having various <a href=\"https:\/\/dataaspirant.com\/word-embedding-techniques-nlp\/\" target=\"_blank\" rel=\"noopener\"><strong>word embedding techniques<\/strong><\/a>. But In this section, we will see how this can be done with the TF-IDF model.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The TF-IDF model scans through the documents one by one and creates word vectors by assigning a weight (relevance score) to each term. While <strong>penalizing<\/strong> the ones which were <strong>very frequent<\/strong> in each and every document and assigning them a <strong>lower<\/strong> score.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Once each word in the document is associated with a number, the model can then be provided with this word vector, and then one can hope from the model to deliver the expected results.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Few <a href=\"https:\/\/dataaspirant.com\/classification-clustering-alogrithms\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>popular machine learning models<\/strong><\/a> used for the smaller text related text are the Navi bias algorithm and simple logistic regression models.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Once we convert the data to numerical, we can use any <a href=\"https:\/\/dataaspirant.com\/category\/machine-learning-2\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>machine learning models<\/strong><\/a> or<a href=\"https:\/\/dataaspirant.com\/category\/deep-learning\/\" target=\"_blank\" rel=\"noopener\"><strong> deep learning models<\/strong><\/a>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Just to give an example, one popular natural language processing, and deep learning algorithms application is text generation. In this case, given some sample words, we are going to create the text.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Here we have to predict the <strong>next word<\/strong>. For this, we use the <a href=\"https:\/\/dataaspirant.com\/how-recurrent-neural-network-rnn-works\/\" target=\"_blank\" rel=\"noopener\"><strong>RNN deep learning model<\/strong><\/a>.<\/p>\n<h2 dir=\"ltr\" id=\"t-1606649429011\">A real-life application of TF-IDF<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">Being able to ascertain the relevance of a term in a document using TF-IDF can be useful in settings like <strong>information retrieval<\/strong>, data mining, search engines, etc.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let\u2019s say we have a company that deals with hotel room images and saves all its information in a database.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">While serving a user who wants to look up rooms with \u201c<strong>stripped wooden flooring<\/strong>,\u201d the search engine has to decide which images will be returned in case none of them match the exact query.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">To make it more intuitive, assume we have to select between the two partial matches:<\/p>\n<p ltr\"\"=\"\">\n<ul>\n<li dir=\"ltr\">Dark wooden flooring<\/li>\n<li dir=\"ltr\">Ruled hardwood flooring<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Even though the first partial match has 2 out of 3 words matching those in the query, we have to understand how TF-IDF can help choose the second option (which we can see is a better match).<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The Term frequency will be the same for each word, but the Inverse document frequency will create all the difference.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">The term <strong>\u201cflooring\u201d<\/strong> will have low IDF values since it would appear in most of the hotel room listings, while the term <strong>\u201cstriped\u201d<\/strong> will have a larger IDF since it would appear in only a few listings and thus has more discriminative power.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">So, we return back the images against the 2nd partial match.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">I hope we have given a clear understanding. Now let\u2019s go to the fun part.<\/p>\n<blockquote dir=\"ltr\" style=\"text-align: center;\"><p><strong>Coding..!<\/strong><\/p><\/blockquote>\n<h2 dir=\"ltr\" id=\"t-1606649429012\">Implementing TF-IDF With Python Code<\/h2>\n<p><span><img decoding=\"async\" alt=\"TF-IDF Implementation In Python\" data-id=\"7545\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"TF-IDF Implementation In Python\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/6-TF-IDF-Implementation-In-Python.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"--smush-placeholder-width: 626px; --smush-placeholder-aspect-ratio: 626\/376;aspect-ratio: auto 750 \/ 450;\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"TF-IDF Implementation In Python\" data-id=\"7545\" width=\"626\" data-init-width=\"750\" height=\"376\" data-init-height=\"450\" title=\"TF-IDF Implementation In Python\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/6-TF-IDF-Implementation-In-Python.png\" data-width=\"626\" data-height=\"376\" loading=\"lazy\" style=\"aspect-ratio: auto 750 \/ 450;\"><\/noscript><\/span><\/p>\n<p style=\"text-align: center;\">TF-IDF Implementation In Python<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Having theoretical knowledge is one thing, but it is even better if one is also able to apply their understanding by implementing the same in code.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let\u2019s look briefly into a python implementation of TF-IDF class and try to understand the code, one block at a time.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">All the <strong>code snippets<\/strong> are taken from <a href=\"https:\/\/github.com\/kumarnikhil936\/TF-IDF\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>Nikhil Github Repo<\/strong><\/a><\/p>\n<\/p>\n<p dir=\"ltr\">Initialize the variables like&nbsp;<\/p>\n<ul>\n<li dir=\"ltr\">number of documents,&nbsp;<\/li>\n<li dir=\"ltr\">list of stopwords,&nbsp;<\/li>\n<li dir=\"ltr\">a dictionary to store the frequency of how many documents a term is present in, etc.&nbsp;<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">If a <strong>corpus file<\/strong> or a <strong>stopword file<\/strong> is supplied along with, call the respective function to absorb this information.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For downloading the <strong>corpus file<\/strong> you can use the below link.<\/p>\n<ul>\n<li dir=\"ltr\" style=\"text-align: justify;\"><a href=\"https:\/\/raw.githubusercontent.com\/gearmonkey\/tfidf-python\/master\/tfidf_testcorpus.txt\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">https:\/\/raw.githubusercontent.com\/gearmonkey\/tfidf-python\/master\/tfidf_testcorpus.txt<\/a><\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Needs to have a specific format where the first line tells about the number of documents.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">While the following lines tell about the number of documents a certain term is present in.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For <strong>downloading<\/strong> the stopword file you can use the below link.<\/p>\n<ul>\n<li dir=\"ltr\" style=\"text-align: justify;\"><a href=\"https:\/\/raw.githubusercontent.com\/gearmonkey\/tfidf-python\/master\/tfidf_teststopwords.txt\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">https:\/\/raw.githubusercontent.com\/gearmonkey\/tfidf-python\/master\/tfidf_teststopwords.txt<\/a>&nbsp;<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">You should have one stopword every line.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Use this function to add a corpus to the existing corpus with the TF-IDF class. Firstly, parse the corpus file with the right settings (mode and encoding).&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Read the first line to fetch the value of the number of documents and add it to the existing value num_docs.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Read the subsequent lines properly to fetch the token name and their frequency value, and all the information to the respective variables. <\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If the token is a known one, add the frequency value to the previously known value, or else create a new entry in the dictionary object.<\/p>\n<\/p>\n<p style=\"text-align: justify;\">Add the data from one new document to the existing information maintained by the class object. Increment the variable <strong>num_docs<\/strong> by 1, and update the frequency values for the individual tokens.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">This function can be used to save the information maintained in the class object in a separate dump file in a file format similar to discussed above for corpus as well as stopwords file.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Firstly open the corpus file in write mode, append the <strong>num_docs<\/strong> value, and then write one token at a time with their frequency values.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Next, open the stopwords file, and then using a threshold value, identify how many (and which) of the tokens qualify as a potential stopword. Write those tokens in the stopwords file.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Determine the value of IDF for a specific token (or term) by using this function. The function first checks if the term is present in the list of tokens flagged as stopwords by the class object and, if so, returns 0.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Next, it checks and returns the frequency value corresponding to that token in the dictionary object <strong>term_num_docs<\/strong>.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">If the token is not present there, a default constant IDF value is returned.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Finally, use the <strong>get_doc_keywords<\/strong> function to calculate the TF-IDF value for the given document. We start with identifying the unique set of tokens from the given document.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For the token set, we calculate TF and IDF values and then later multiply the two to get the TF-IDF value. The results returned are ordered by tokens having highest to lowest relevance.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Let\u2019s run a small test script to verify the functionality.<\/p>\n<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Initially, we are creating an object of the class TfIdf. Further, we read the corpus and stopwords file (Observe in the output of how the data is actually stored).&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Lastly, we calculate the TF-IDF values for every word in the corpus. Since we are working with only a single file in this example, the value of TF for every word in the same 1.0, which can be seen more in detail in the excerpt below.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Also, the words marked as stopwords will have a <strong>TF-IDF value of 0<\/strong>.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Below is the output.<\/p>\n<\/p>\n<p dir=\"ltr\">You can get the<strong> complete<\/strong> code in our <a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/tree\/master\/tf-idf\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>GitHub repo<\/strong><\/a>. For all our Dataaspirant code please visit <a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\"><strong>Dataaspirant Github<\/strong><\/a> repo.<\/p>\n<h2 dir=\"ltr\" id=\"t-1606649429013\">What next?<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">While it is actively used in NLP related problems. TF-IDF can also prove to be a game-changer in establishing a refined content publishing strategy and further improving the search engine optimizations (SEO).<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">Running a TF-IDF analysis on your own website (or on a competitor\u2019s website) might help discover some gaps which should be immediately fixed or discover some concepts which were never explored before.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">For the web pages bringing the most traffic, it is important to know if they are the best version of themselves, and then work on further improving the content.<\/p>\n<h2 dir=\"ltr\" id=\"t-1606649429014\">Conclusion<\/h2>\n<p dir=\"ltr\" style=\"text-align: justify;\">We know for a fact that machine learning algorithms work better with numerical data, and the TF-IDF algorithm acts as a bridge to connect such algorithms with the text data.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\"><strong>Text vectorization<\/strong> has played a very important part in NLP areas like<\/p>\n<ul>\n<li dir=\"ltr\" style=\"text-align: justify;\">sentiment analysis,&nbsp;<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\">information retrieval,<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\">text analysis,<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\">search engine optimization,<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\">text summarization,<\/li>\n<li dir=\"ltr\" style=\"text-align: justify;\">ranking the results to a query, etc.<\/li>\n<\/ul>\n<p dir=\"ltr\" style=\"text-align: justify;\">Having a good understanding of <strong>how TF-IDF works<\/strong> will indeed also help gain clarity on how the various machine learning algorithms functions.<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">While we looked at one of the simplest ways of working with TF-IDF by summing the tf-idf for each query term, many more sophisticated ranking functions are variants of this simple model.<\/p>\n<h4>Recommended Natural Language Processing Courses<\/h4>\n<p>Recommended<span style=\"\"><img decoding=\"async\" alt=\"nlp specialization\" data-id=\"4068\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"NLP specialization\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/07\/nlp-spelization-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"--smush-placeholder-width: 176px; --smush-placeholder-aspect-ratio: 176\/176;aspect-ratio: auto 150 \/ 150;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"nlp specialization\" data-id=\"4068\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"NLP specialization\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/07\/nlp-spelization-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"aspect-ratio: auto 150 \/ 150;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>NLP Specialization with Python<\/h4>\n<p style=\"\">Rating: <strong>4.5\/5<\/strong><\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"Deep-learning-for-computer-vision2.png\" data-id=\"6739\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"Deep-learning-for-computer-vision2.png\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/10\/Deep-learning-for-computer-vision2.png-150x150.jpeg\" data-width=\"176\" data-height=\"176\" style=\"--smush-placeholder-width: 176px; --smush-placeholder-aspect-ratio: 176\/176;aspect-ratio: auto 150 \/ 150;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"Deep-learning-for-computer-vision2.png\" data-id=\"6739\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"Deep-learning-for-computer-vision2.png\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/10\/Deep-learning-for-computer-vision2.png-150x150.jpeg\" data-width=\"176\" data-height=\"176\" style=\"aspect-ratio: auto 150 \/ 150;\" mt-d=\"-64\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Deep Learning Course<\/h4>\n<p style=\"\">Rating: <strong>4.5\/5<\/strong><\/p>\n<p><span style=\"\"><img decoding=\"async\" alt=\"natural-language-processing-python\" data-id=\"4072\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"natural-language-processing-python\" data-src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/07\/natural-language-processing-python-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"--smush-placeholder-width: 176px; --smush-placeholder-aspect-ratio: 176\/176;aspect-ratio: auto 150 \/ 150;\" mt-d=\"-19\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\" src=\"data:image\/gif;base64,R0lGODlhAQABAAAAACH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==\" class=\"lazyload\"><noscript><img decoding=\"async\" alt=\"natural-language-processing-python\" data-id=\"4072\" width=\"176\" data-init-width=\"150\" height=\"176\" data-init-height=\"150\" title=\"natural-language-processing-python\" src=\"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/07\/natural-language-processing-python-150x150.jpg\" data-width=\"176\" data-height=\"176\" style=\"aspect-ratio: auto 150 \/ 150;\" mt-d=\"-19\" center-v-d=\"false\" mt-t=\"-65\" mt-m=\"-86\" loading=\"lazy\"><\/noscript><span style=\"\"><\/span><\/span><\/p>\n<h4>Machine Learning Course<\/h4>\n<p style=\"\">Rating: <strong>4\/5<\/strong><\/p>\n<h4>About the Author: Nikhil Kumar<\/h4>\n<p dir=\"ltr\" style=\"text-align: justify;\"><a href=\"https:\/\/www.linkedin.com\/in\/nikhil-kumar-jha\/\" target=\"_blank\" style=\"outline: none;\" rel=\"noopener\">Nikhil<\/a> is a creative and product-focused Data Scientist who likes to work on the most challenging as well as productive machine learning, and big data use cases.&nbsp;<\/p>\n<p dir=\"ltr\" style=\"text-align: justify;\">He has eloquent experience while working with real-world challenges such as anomaly\/intrusion detection, <strong>dynamic pricing<\/strong>, predicting customer intents, ranking search results, time-series classification, deriving insights from unstructured data such as images and text, making personalized recommendations, and innovating the user experience.<\/p>\n<p><span style=\"font-size: 14px;\">Previously he did his <strong>Masters of Science in Computer Engineering<\/strong> with a specialization in Signal, Image, and Speech Processing from the <strong>University of Paderborn<\/strong> (Germany)<\/span><\/p>\n<h4>Follow us:<\/h4>\n<h5><strong><span style=\"color: #0000ff;\"><a style=\"color: #0000ff;\" title=\"dataaspirant\" href=\"https:\/\/www.facebook.com\/dataaspirant\" target=\"_blank\">FACEBOOK<\/a><\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"dataaspirant.quora.com\" href=\"http:\/\/dataaspirant.quora.com\/\" target=\"_blank\">QUORA<\/a><\/span>\u00a0|<span style=\"color: #00ccff;\"><a style=\"color: #00ccff;\" title=\"twitter.com\/dataaspirant\" href=\"https:\/\/twitter.com\/dataaspirant\" target=\"_blank\">TWITTER<\/a><\/span>|\u00a0<span style=\"color: #808000;\"><a style=\"color: #808000;\" href=\"https:\/\/plus.google.com\/104087213513583475222\" target=\"_blank\">GOOGLE+<\/a><\/span>\u00a0|\u00a0<span style=\"color: #000080;\"><a style=\"color: #000080;\" href=\"https:\/\/www.linkedin.com\/company\/dataaspirant\" target=\"_blank\">LINKEDIN<\/a><\/span>|\u00a0<span style=\"color: #ff9900;\"><a style=\"color: #ff9900;\" title=\"reddit dataaspirant\" href=\"http:\/\/www.reddit.com\/user\/dataaspirant\/\" target=\"_blank\">REDDIT<\/a>\u00a0<\/span>|\u00a0<span style=\"color: #ff0000;\"><a style=\"color: #ff0000;\" title=\"flipboard dataaspirant\" href=\"https:\/\/flipboard.com\/@dataaspirant\/\" target=\"_blank\">FLIPBOARD<\/a><\/span>\u00a0|\u00a0<span style=\"color: #00ff00;\"><a style=\"color: #00ff00;\" title=\"medium dataaspirant\" href=\"https:\/\/medium.com\/@dataaspirant\" target=\"_blank\">MEDIUM<\/a>\u00a0|\u00a0<a href=\"https:\/\/github.com\/saimadhu-polamuri\/DataAspirant_codes\" target=\"_blank\"><span style=\"color: #008080;\">GITHUB<\/span><\/a><\/span><\/strong><\/h5>\n<p style=\"text-align: justify;\">I hope you like this post. If you have any questions ? or want me to write an article on a specific topic? then feel free to comment below.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For building any natural language model, the key challenge is how to convert the text data into numerical data. As the machine learning or deep learning models don\u2019t understand the text data. One smart way to do the conversion is the TF-IDF method. This TF-IDF method is a popular word embedding technique used in various<br \/><a class=\"moretag\" href=\"https:\/\/dataaspirant.com\/tf-idf-term-frequency-inverse-document-frequency\/\">+ Read More<\/a><\/p>\n","protected":false},"author":19,"featured_media":7517,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":"","_jetpack_memberships_contains_paid_content":false},"categories":[2049],"tags":[2110,2048],"aioseo_notices":[],"jetpack_sharing_enabled":true,"jetpack_featured_media_url":"https:\/\/dataaspirant.com\/wp-content\/uploads\/2020\/11\/1-How-TF-IDF-works.png","_links":{"self":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/7515"}],"collection":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/users\/19"}],"replies":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/comments?post=7515"}],"version-history":[{"count":6,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/7515\/revisions"}],"predecessor-version":[{"id":11868,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/posts\/7515\/revisions\/11868"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media\/7517"}],"wp:attachment":[{"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/media?parent=7515"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/categories?post=7515"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dataaspirant.com\/wp-json\/wp\/v2\/tags?post=7515"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}