skonkiel/mars_news

Scraping data on Mars, serving it up via a Flask-powered application.

Mars News

Project background

This application includes scraped data from a series of websites, stored locally in a MongoDB database, then repurposed into a webpage that contains facts and images about Mars.

This Flask app uses splinter (Selenium), MongoDB, Beautiful Soup and Pandas. It was designed using Bootstrap.

What's in this repo

• A Jupyter notebook, including Python code to scrape the data and store it in a MongoDB database
• scrape_mars.py, containing the executable code for use in the Flask app
• A Flask app that powers a webpage to display the data
• A templates folder that contains the webpage

mars_news's Issues

Fix error caused by NASA website / add try-except error handling

Behavior: Page loads OK initially, but when you press "Scrape data" button, the app breaks because the NASA page data isn't being scraped properly, and there's no try/except error handling to return a user-friendly error message.

Notes from discussion with Garrett:

• The higher up in the DOM, the less likely it is that the attribute ID/class will change
• Use Try/Except/Finally
• Use loops to look for content in page (get all li where class=“slide”, then return everything in it; if class = specific div I’m looking for, then do the thing. Otherwise, return error message to API (so user doesn't see it).