aimeevu/web-scraping-challenge

GA Tech Data Science and Analytics Boot Camp Module 12

web-scraping-challenge

GA Tech Data Science and Analytics Boot Camp Module 12

Description

This module utilizes new concepts, such as web scraping with BeautifulSoup and MongoDB, and former concepts like HTML, Bootstrap, and Flask to develop an application that will scrape data from four sites and display the data through a Flask application.

Part 1: Scraping

This module first starts with a Python Jupyter notebook in order to utilize BeautifulSoup, Pandas, and Requests/Splinter to scrape for data.

The four sites that data is scraped from include:

• NASA Mars News
  • Collect the latest News Title and Paragraph Text

• JPL Marks Space Images-Featured Image
  • Save complete URL string for the featured image

• Mars Facts
  • Collect table containing facts about Mars

• Mars Hemispheres
  • Save image URLs for each hemisphere, including the title

The code for the scraping script can be found here.

Part 2: MongoDB and Flask Application

Once the script works in Jupyter, it is then copied into a Python file in order for the script to be used by the Flask application. This code can be found in scrape_mars.py

The Flask application will then call to the scraping script in order to load and display the data in the browser. At the same time, the data is loaded into a MongoDB database named marsDataDB, collection marsData.

Additionally, an HTML template that utilizes Bootstrap is called into the Flask application in order to properly display the data.

The code for the Flask application can be found here, and the HTML template can be found here.

Here is the final application:

Submission Requirements

• Jupyter notebook with scraping code
• Screenshots of final application
• Flask App with HTML template