Keen Learner and Exp... • 1m
Day 3 of learning AI/ML as a beginner. Topic: NLP (Tokenization) Tokenization is breaking paragraph (corpus) or sentence (document) into smaller units called tokens. In order to perform tokenization we use nltk (natural language toolkit) python library. nltk is not a built in library and therefore needed to be installed locally in the desktop. Therefore I first used pip to install nltk and the from nltk I imported all those things which I needed in order to perform tokenization. I required sent_tokenize, word_tokenize, wordpuct_tokenize and TreebankWordTokenizer. Sent_tokenize: this breaks a corpus (paragraph) into document (sentences). Word_tokenize: this breaks a document into words. Wordpunct_tokenize: this does the same thing as word tokenize however this also considers punctuations ("'" "." "!" etc). TreebankWordTokenizer: This does not assume "." as a new word, it assumes it a new word only when it is present with the very last word. And here's my code and it's result. I warmly welcome all the suggestions and questions regarding this as they will help me deepen up my knowledge while also help me improve my learning process.
Keen Learner and Exp... • 1m
Day 4 of learning AI/ML as a beginner. Topic: text preprocessing stemming using NLTK. I have learned about tokenization and now I am learning about text preprocessing in ML. Text preprocessing is cleaning up of raw text (raw text is the one entered
See MoreKeen Learner and Exp... • 1m
Day 12 of learning AI/ML as a beginner. Topic: TF-IDF practical. Yesterday I shared my theory notes and today I have done the practical of TF-IDF. For the practical I reused my spam classifier code and for TF-IDF I first imported it from the sklear
See MoreHii I am Dhoni saroj... • 9m
1. Concept 💡 The Transparent Kitchen features a glass-enclosed kitchen where customers can watch their meals being prepared in real-time. 2. Transparency 🔍 This transparency builds trust by ensuring high food quality, hygiene, and cleanliness durin
See MoreDownload the medial app to read full posts, comements and news.