sara-zinnat/natural-language-processing-projects

Natural-Language-Processing-Projects

These are the projects that I have done while doing my NLP course.

Project-1

File Name : Token word count.py

The aim of this project is to count word tokens, word types from each genre/category of the Brown corpus. Additionally, the vocabulary size of the whole corpus is also calculated. This project contains the following segments:

1. removing special character and lower case conversion
2. counting tokens
3. removing stopwords
4. applying lemmatization
5. applying stemming
6. counting word type
7. main

Project-2

File Name: Sent gen Bi Tri gram.py

In this project the goal is to generate random sentences using Bi-gram and Tri-gram approaches. This project contains the following segments:

1. removing special character and lower case conversion
2. removing stopwords
3. tokenizing the entire brown corpus incorporating tokens for starting and ending of each sentences
4. make an uni-gram table
5. make a bi-gram table
6. make a tri-gram table
7. generate and print a random sentence using bi-gram
8. generate and print a random sentence using tri-gram
9. main

Project-3

File Name: Text classification LogRef Multi Layer.py

This project aims to classify text using Logistic regression and Multi-layer neural network on Brown corpus. This project contains the following segments:

1. removing special character and lower case conversion
2. removing stopwords
3. applying lemmatization
4. generates features matrix and labels
5. main

Project-4

File Name: Text classification Naive Bayes.py

This project aims to classify text using Naive Bayes classifier on Brown corpus. This project contains the following segments:

1. removing special character and lower case conversion
2. removing stopwords
3. applying lemmatization
4. generates features matrix and labels
5. main

Project-5

File Name: Cosine similarity.py

This program aims to measure cosine similarity within and between the clusters. This project contains the following segments:

1. fetch similarity between two sentences from matrix (using tf-idf and word2vec)
2. to calculate similarity between two documents (using tf-idf and word2vec)
3. main