guada-aguero/lab-string-operations

Lab | String Operations and Bag of Words

Introduction

In this lab, we will learn how to manipulate strings. There are two challenges: 1) to practice how to manipulate strings, and 2) to use string manipulation techniques to create Bag of Words (BoW). BoW is an essential technique in Natural Language Processing.

Getting Started

In your Terminal, navigate into the directory your-code of this lab that contains challenge-1.ipynb, challenge-2.ipynb, doc1.txt, doc2.txt, and doc3.txt. Start Jupyter Notebook by executing jupyter notebook. A webpage should automatically open for you but in case not, go to http://localhost:8888. Then click the link to each ipynb file to complete the challenges.

Deliverables

challenge-1.ipynb and challenge-2.ipynb with your responses.

Submission

Upon completion, add your deliverables to git. Then commit git and push your branch to the remote.

Resources

• The re Library

• F-strings

• Regular Expressions

• Python Input and Output (how to read file content)

• How to Remove Punctuation in Python String

• Convert String to Lowercase in Python

• Break Python String into Array

• What is Text Corpus?

• A Gentle Introduction to the Bag-of-Words Model

Additional Reading

If you are a research-type person, you will find this article interesting. Scientists used techniques based on BoW to calculate the frequency of words used cross 17 world languages. They found there is a consistent pattern in terms of the frequency of words being used in human languages. Some mad scientists even want to use this technique to analyze dolphin language because they believe they can build corpora based on the sounds dolphins make, correlate the dolphin language corpora with human language corpora, and potentially understand what dolphins speak. 😲 😲 😲

Data analytics is now entering almost every discipline and profession. You will want to reflect on how you will apply your data analytics skills to the fields you are familiar with -- in creative ways. There are tons of fun secrets waiting for you to discover with data analytics.