aarthy2000/kaggle_bag-of-words-meets-bags-of-popcorns

A experiment for Bag of Words Meets Bags of Popcorn on Kaggle (https://www.kaggle.com/c/word2vec-nlp-tutorial)

This is a repo for Bag of Words Meets Bags of Popcorn on Kaggle

Experiment Flow

• Data preprocessing
• Generate word2vec data
• Train a CNN model for text classification
• Predict testing data

Data preprocessing

• Observe data format
• Statistics
• Data cleaning
• word2vec

Preprocessing steps of data cleaning

1. Remove the " character at the start/end of review
2. Replace \" to "
3. Replace multiple <br /> to space character
4. Tokenize the whole review (article)
5. Lemmatize all words

Preprocessing steps of generating word2vec files

1. Remove the " character at the start/end of review
2. Replace \" to "
3. Replace multiple <br /> to space character
4. Split sentences from review (article)
5. Tokenize all sentences
6. Lemmatize all words

Convolution Neural Network (CNN) Model

1. Preprocessing for CNN training data
2. Training model
3. Test model by validation set
4. Find out a good model
5. Generate predict by the good model

preprocessing for CNN training data

1. Split data into training set and validation set
2. Index all words in word2vec
3. Encode all the words of reviews in training data and validation data