gtmaskall/dsc_intro

Exercises to accompany the free Springboard introductory data science "taster" course.

Springboard data science taster course notebooks

worked_notebooks

Contain the worked example notebooks that include completed code.

source_notebooks

These are sourced from worked_notebooks but have had the assignment code deleted and output cleared. Students will load these notebooks and work from them. They can, if and when they choose, refer to the original notebooks in worked_notebooks to check their answers or get unstuck.

About me

Hi, I'm Guy. I'm a mentor with Springboard and really hope you find this taste of learning data scientist is something you'd like more of. If you have any questions, or are thinking of signing up to take things further with Springboard's full courses, then drop me a line. I'll be happy to help.

Yelp dataset

Note, these exercises were put together using Round 12 of Yelp's challenge. Currently Yelp are on round 13. I haven't revisited the exercises using the newer dataset. I hope you still get sensible answers, but let me know if this causes you any difficulties. If you'd like the original dataset (from round 12) let me know. If this option is popular I may devote some of my Google drive to hosting it.

dsc_intro's Issues

TypeError when reading review.json dataset (05_case_study_filter_reviews.ipynb)

When I first set up the code to read in the review.json dataset, I encountered an error:

TypeError: sequence item 0: expected str instance, bytes found

Code

path 'file://localhost/Users/.../DSC_Intro/'
filename = path + 'yelp_dataset/review_Test.json'
review_reader = pd.read_json(filename, lines=True, chunksize=100000)

for chunk in review_reader:
    print(chunk)
...

I changed the code to use

from io import StringIO
...
review_reader = pd.read_json(StringIO(filename), lines=True, chunksize=100000)

insead and this error went away.

Referencing panda users guide indicates that the StringIO function is required.

You may want to update the worked notebook accordingly.

ValueError when reading review.json dataset (05_case_study_filter_reviews.ipynb)

After adding StringIO(...) to the reader creation code in 05_case_study_filter_reviews.ipynb. I am encountering a new error when trying to read the review file in chunks.

ValueError: Unexpected character found when decoding 'false'

N.B.

• The error only occurs when trying to read the data from a file on disk
  • If the records are placed inline within a string or "here doc" (``` string), the error does not occur
• The error only occurs when trying to read the file in chunks.
  • If the complete data file is read in, without using chunksize to create a reader, there is no error.
• I have reduced the size of the data set for testing. I can reproduce the error with a single JSON record in the file.

Sample Data

Either or both of the following records can be used

{"review_id":"rEITo90tpyKmEfNDp3Ou3A","user_id":"6Fz_nus_OG4gar721OKgZA","business_id":"6lj2BJ4tJeu7db5asGHQ4w","stars":5.0,"useful":0,"funny":0,"cool":0,"text":"We've been a huge Slim's fan since they opened one up in Texas about two years ago when we used to live there. This place never disappoints. They even have great salads and grilled chicken. Plus they have fresh brewed sweet tea, it's the best!","date":"2017-05-26 01:23:19"}
{"review_id":"Amo5gZBvCuPc_tZNpHwtsA","user_id":"DzZ7piLBF-WsJxqosfJgtA","business_id":"qx6WhZ42eDKmBchZDax4dQ","stars":5.0,"useful":1,"funny":0,"cool":0,"text":"Our family LOVES the food here. Quick, friendly, delicious, and a great restaurant to take kids to. 5 stars!","date":"2017-03-27 01:14:37"}

Error Output

I have filed an issue with pandas-dev as well as a question on stackexchange, but haven't gotten very far.