marketulinek/webbinspace

What is the Webb observing right now? 🌌

My personal project I am developing while exploring Django and scraping in Python 🔭

Webb In Space

It's showing what the Webb telescope is observing right now.

The application is divided into three parts:

Scraping the data:

• It scrapes this page looking for new reports and saved them into /data_source/cycle_{number}/ folder as new txt file.
• Use command: python manage.py observation_plan_scout.

Parsing the data:

• It searches the data_source folder looking for new files. Then parses the data in these files and saves them into the database.
• Use command: python manage.py report_parser.

Showing the data on the web page:

• Homepage shows what is the current target of the Webb's observation.

How to install

🐍 Python 3.9 required

• Install the dependencies: pip install -r requirements.txt
• Apply migrations: python manage.py migrate
• Create your .env file:

# Example for development
DEBUG=True
DEVELOPMENT_MODE=True
ALLOWED_HOSTS=127.0.0.1,localhost

• Run the app: python manage.py runserver

Built With

• Django - Python web framework
• Bootstrap - Front-end framework
• Chart.js - JavaScript charting library
• Mermaid - JavaScript-based diagramming and charting tool

webbinspace's Issues

Switch to docker-compose

New command: invalidate reports that contain no visit

Add chart to show the time spent on observing Solar system

Keywords are not scraped in full

Handle 404 or 500 error

https://docs.djangoproject.com/en/4.1/howto/error-reporting/

Maintenance mode

https://github.com/bashu/django-maintenancemode looks good

https://github.com/fabiocaccamo/django-maintenance-mode better

Create a script to update keywords for existing visits

Sitemap

https://docs.djangoproject.com/en/4.1/ref/contrib/sitemaps/

Adapt scraper to new HTML of the target site

Add cycle 2 to charts

Report list

Create a page with list of reports to show the number of visits for each report.

Needed for a better quick overview which reports has zero number of valid visits. This breaks the scraper and it's been happening a lot lately.

• Table of reports
• Count number of visits for each report
• Distinguish those reports that have zero records
• Write tests

Group dependabot bumps into one bundle

Test improvements

To improve tests with the knowledge I've learned.

Media query not working on mobile

It works fine in Firefox but not on real mobile.

Possible solution:
https://stackoverflow.com/questions/17344339/media-query-not-working-in-mobile-works-fine-in-chrome

Report file can have the same package number as another report

Add favicon

Add a mention of the Chart.js library to the Built With section in the README

Add a new column 'file_name' to Report model and fill it with values

• Update model
• Create migration
• Update docs
• Create script that updates data (file name, package number - latest reports have wrong value)