Giter Site home page Giter Site logo

code4romania / seismic-risc Goto Github PK

View Code? Open in Web Editor NEW
34.0 9.0 67.0 15.36 MB

Web app for keeping track of buildings in danger of collapsing in the event of an earthquake

Home Page: https://seismic-risc.vercel.app/

License: Mozilla Public License 2.0

Python 38.87% HTML 1.84% JavaScript 50.34% Makefile 3.02% Shell 0.78% SCSS 5.08% Batchfile 0.02% CSS 0.05%
civic-tech crowdsourcing earthquakes code4ro django react python javascript

seismic-risc's Introduction

Acasă în Siguranță

GitHub contributors GitHub last commit License: MPL 2.0

code for romania twitter code for romania facebook code for romania instagram code for romania linkedin

🇷🇴 Un cutremur în București nu este o situație ipotetică. Este o certitudine că acest lucru se va întâmpla. În acest context, la mai bine de 40 de ani de la cutremurul din 1977, memoria colectivă a ascuns în profunzime amintirile acelui dezastru în încercarea de a-și înăbuși teama. Dar realitatea este că, patru decenii mai târziu, Bucureștiul, la fel ca restul orașelor cu risc seismic ridicat, nu ar face față unui asemenea eveniment, iar pierderile de vieți omenești ar fi uriașe. Exercițiul Seism 2018 derulat de DSU arată că cel puţin 4.587 persoane şi-ar pierde viaţa, iar 8.585 ar fost rănite, 6 spitale vor fi distruse, 23 de unităţi spitaliceşti distruse parţial, iar 9 avariate, dar funcţionale. O estimare, am spune noi, destul de optimistă.

Ce putem face pentru a deveni mai puțin vulnerabili? Să știm totul despre oraș, despre clădirile în care locuim astfel încât să putem cere consolidarea lor. Acasă în Siguranță nu este doar "un nou site de informare", ci o platformă care colectează și validează apoi cu experți date despre clădirile din România, la nivel național, ajută asociațiile de proprietari să își consolideze clădirile, te ține la curent cu legislația și ți-o explică și are grijă să ai la îndemână informații utile la orice moment.

🇬🇧 An earthquake in Bucharest is not a hypothetical situation. It is certain that this will happen. In this context, after more than 40 years from the 1977 earthquake, the collective memory has hidden deep the memories of that disaster in its attempt of stifling its fear. The reality is that, four decades later, Bucharest, as well as the rest of the cities with a high seismic risk, would not stand up tu such an event, and the loss of life would be tremendous. The Earthquake 2018 Exercise conducted by the DSU shows that at least 4,587 people would have died and 8,585 would have been injured, 6 hospitals would be destroyed, 23 more would be partially destroyed, and 9 would be damaged, though still functional. An estimation that we would consider quite optimistic.

What can we do to become less vulnerable? Find out everything about the city, about the buildings in which we live so that we can ask for their consolidation. Home Safe is not just "a new information site", but a platform that collects and then validates with the help of experts data about the buildings in Romania, at a national level, it helps owners associations to consolidate their buildings, it keeps you in touch with the current legislation and explains it to you, and it makes sure that you have useful information at your disposal at all times.

Let's save lives together.

TABLE OF CONTENTS (click to expand)

Contributing

If you would like to contribute to one of our repositories, first identify the scale of what you would like to contribute. If it is small (grammar/spelling, or a bug fix) feel free to start working on a fix. If you are submitting a feature or substantial code contribution, please discuss it with the team and ensure it follows the product roadmap.

Our collaboration model is described here. And make sure you check the workflow document; it helps you keep your environment in a good shape, and it helps everyone move faster with code reviews. If you want to make any change to this repository, please make a fork first.

We don't have a specific set of coding guidelines, so just follow the way the code was written until now, if in doubt, you can use Google's style guide.

Built With

Programming languages

Python 3 JavaScript

Frameworks

Backend: Django Client: React

Package managers

Backend: pip Client: npm

Code styling

Backend: Black Client: Prettier + ESLint + Airbnb style guide

Database technology & provider

PostgreSQL

Getting started

Risc Seismic backend is a Django application, built on top of Python 3.9+ with a PostgreSQL database. The Client is a React single page application.

Pre-requisites

In order to run the project locally, you need to have Docker and docker-compose installed.

You can install the above-mentioned packages manually, or you can use our helper commands.

On Ubuntu run:

make install-docker-ubuntu

On MacOS run:

make install-docker-osx

On other platforms please follow the instructions described here:

Initial set-up

Initialising all the services

Make sure to check the Environment variables section for info on how to set up the keys before you run the following commands:

cp .env.example.dev .env.dev
# build the development container
make build-dev

If you didn't set up the RUN_LOAD_INITIAL_DATA variable, you can add dummy data to the database with the following command:

make build-dev

If the RUN_LOAD_INITIAL_DATA was yes, then you should have dummy data but will have to create a superuser:

docker-compose exec api ./manage.py createsuperuser

Backing up and restoring the database in case of an upgrade

  1. Create a back-up of the data (the build folder is ignored by git)

    docker-compose exec db pg_dumpall -U postgres > ./build/backup.sql
  2. Run the database upgrade or just get the latest version of the code from git if it upgrades the database

    git pull upstream develop
  3. Remove the current database and start-up the environment (remove the -dev part if you don't want the development mode)

    make drop-db && make build-dev
  4. Restore the backed-up data to the new database

    docker-compose exec db psql -U postgres < ./build/backup.sql
  5. Check the API endpoints and that you can log in the admin interface with the same users as before

Environment variables

Deployment variables

The following variables change the way the backend is deployed.

RUN_MIGRATIONS Run the initial migrations (sets up the data models from the database).

RUN_LOAD_INITIAL_DATA Adds real & dummy data to the database (adds buildings, datafiles, and statistics).

RUN_COLLECT_STATIC Collects static files so that they can be easily served to production.

RUN_DEV_SERVER Runs the application in the development mode.

External services API keys

To have a fully functional project, you have to get two API keys: HERE Maps API Key and hCAPTCHA API Key.

HERE Maps API Key

Tutorial: https://developer.here.com/tutorials/getting-here-credentials/

Keys added to the .env file:

# the same key can be used for both variables
HERE_MAPS_API_KEY
REACT_APP_HERE_MAPS_API_KEY
hCAPTCHA API Key
  1. Create a hCAPTCHA account
  2. Go to your settings page
  3. Create a New Site, copy the Site Key and add it to the environment variables list

Keys added to the .env file:

REACT_APP_CAPTCHA_API_KEY

Starting the project

First check the .env file created by the init command and see if there are any environment variables that you might need to provide or change. This file is used by docker-compose to pass the environment variables to the container it creates.

Get the project up and running:

docker-compose up

You should be able to access the local environment site and admin at the following URLs:

If you have problems starting the project, first check out the FAQ and if that doesn't work, ask someone from the project's channel. Maybe the issue you just had is worth adding to the FAQ, wouldn't it?

To work on running containers that were started using docker-compose up, open another terminal and:

cd path/to/repo
docker-compose exec api some_container_command
# or
docker-compose exec client some_container_command

To see all available commands, run:

make help

Starting the project without Docker

Windows platform

Prerequisites
  1. PostgreSQL
  2. Python 3.9
  3. Node.js
Steps to set your environment
  1. In project directory run:

    python -m venv .venv
    .venv\Scripts\activate.bat
    pip install -r ./backend/requirements-dev.txt
    copy .env.dev .env
  2. Check the .env file created by the copy command and see if there are any environment variables that you might need to provide or change. Double check database config line in .env. It has to follow this pattern: postgres://USER:PASSWORD@HOST:PORT/NAME

  3. Run following to set the needed environment variables:

    activate_dev_env.bat
  4. Check database connection. If this fails double check database configuration.

    python backend/wait_for_db.py
  5. Run migrations:

    python backend/manage.py migrate --no-input
  6. Create admin user (user to login into admin panel):

    python backend/manage.py createsuperuser
  7. Load dummy data in database:

    python backend/manage.py loaddata statistics
    python backend/manage.py loaddata buildings
    python backend/manage.py loaddata pages
  8. Install node modules.

    cd client
    npm install

Steps needed to start development servers

1. Start backend server.

Open terminal in the project directory and run environment activation script, then start the server.

.venv\Scripts\activate.bat
activate_dev_env.bat
python backend\manage.py runserver 0.0.0.0:8030

Check functionality at http://localhost:8030 you should get a 404 page.

2. Start front-end server.

Open terminal in the project directory and run environment activation script, then start the server.

activate_dev_env.bat
cd client
npm start

Check functionality at http://localhost:3000.

Development

When creating new models in Django, to make sure they are generated in a clean environment, it is recommended to generate the migration files using the make command:

make makemigrations && make migrate

When you need to add/remove requirements or restrict the version of a requirement, edit the requirements.in (prod) and the requirements-dev.in (dev) files accordingly. After doing this run:

make update-requirements

This will create a clean environment where it uses the pip-tools library to compile the corresponding requirements.txt files with the versions of the packages pinned. This is important as it guarantees that every environment this service runs in, has the same dependencies installed and minimizes the risk of works on my machine.

Known Issues

Client hot-reload on Windows Docker is not working

Try following these steps:

  1. open up a terminal in seismic-risc_client container

  2. cd ./node_modules/react-scripts/config/

  3. vi webpackDevServer.config.js

  4. on the exported config object, update the value of watchOptions to include the following properties:

    aggregateTimeout: 100,
    poll: 500
  5. save the file and restart the client container

This way, webpack-dev-server should be watching files in polling mode, instead of listening for file change events.

In VS Code, ESLint fails to load the Prettier plugin

Add the following option to user settings in VS Code if ESLint fails to load the Prettier plugin.

{
    "eslint.workingDirectories": [
        {
            "mode": "auto"
        }
    ]
}

Management Commands

The new custom command can be called using python manage.py buildings <number> required arguments:

  • --delete
  • --create
cd path/to/repo
docker-compose exec api bash
root@ba4fd81f9023:/code# python manage.py buildings 30 --create
100% |███████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:00<00:00, 37.89it/s]
Successfully created 30 buildings.
root@ba4fd81f9023:/code# python manage.py buildings 25 --delete
Successfully deleted 25 buildings.

Testing

Local development testing:

cd path/to/repo
docker-compose exec api bash
root@3c5df91778ad:/code# pytest

Pipeline testing:

make test

Production

To get the container ready for production use, we need to first build it:

docker build -t seismic-risc:latest ./backend

Use the prod.env.dist template file and create a prod.env file with the correct environment variables and run like so:

docker run --env-file prod.env -p HOST_PORT:GUNICORN_PORT seismic-risc:latest

Or, you can provide all the environment variables at runtime:

docker run -e DJANGO_CONFIGURATION=Prod -e DJANGO_SECRET_KEY= -e DATABASE_URL=postgres://USER:PASSWORD@HOST:PORT/NAME -e GUNICORN_PORT=5000 -e GUNICORN_WORKERS_COUNT=2 -p HOST_PORT:GUNICORN_PORT seismic-risc:latest

After testing the container runs properly, tag and upload the self to Docker hub:

docker tag seismic-risc:latest code4romania/seismic-risc:latest
docker push code4romania/seismic-risc:latest

Client Deployment

  • Change directory to ./client
  • Build the solution npm install
  • Start a development server npm start
  • Run the tests npm test
  • Build the solution npm run build

Feedback

  • Request a new feature on GitHub.
  • Vote for popular feature requests.
  • File a bug in GitHub Issues.
  • Email us with other feedback [email protected]

License

This project is licensed under the MPL 2.0 License — see the LICENSE file for details

About Code4Ro

Started in 2016, Code for Romania is a civic tech NGO, official member of the Code for All network. We have a community of over 500 volunteers (developers, UX/UI, communications, data scientists, graphic designers, devops, IT security, and more) who work pro bono for developing digital solutions to solve social problems. #techforsocialgood. If you want to learn more details about our projects visit our site or if you want to talk to one of our staff members, please e-mail us at [email protected].

Last, but not least, we rely on donations to ensure the infrastructure, logistics and management of our community that is widely spread across 11 timezones, coding for social change to make Romania and the world a better place. If you want to support us, you can do it here.

seismic-risc's People

Contributors

alinamihai avatar aniri avatar aramboi avatar calindragomir avatar catileptic avatar cipick avatar cnstlungu avatar costibleotu avatar danichim avatar danmic94 avatar danniel avatar dependabot-preview[bot] avatar dependabot[bot] avatar emanuelbesliu avatar francisc-czobor avatar gabrielmajeri avatar geozaur avatar imgbot[bot] avatar imgbotapp avatar iuliux avatar juliageek avatar justbeyou avatar kata-kas avatar mateesville93 avatar myshy93 avatar raducstefanescu avatar razvanpavel avatar sk-anna avatar tudoramariei avatar vladplesu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

seismic-risc's Issues

Implement Django REST framework

To have a more streamlined communication between front-end and back-end, a better, more flexible API should be created to handle the information

[frontend] Create detailed building view

As a user, when I click on a building, I want to see a side-view on the right-hand side of the map where I can read all the details about the currently selected building, in order to understand the context in which it was marked as being a risk.

As reference, use the second mockup image.

Data model refactoring

The data model of the application needs to be refactored:

  • translate data model to english
  • make the data model more general, as currently the model is mostly designed for Bucharest

Internationalise the interface

All the text displayed should be multilingual, allowing us to deploy the application in different languages
This also allows the display of Romanian strings while at the same time allowing developers to easily understand what those strings represent

Languages supported: en-GB (code language), ro-RO

Mock text can be used in the beginning to not spend much time on translating everything

Relevant React i18n docs:

Internationalise the interface

All the text displayed should be multilingual, allowing us to deploy the application in different languages
This also allows the display of Romanian strings while at the same time allowing developers to easily understand what those strings represent

Languages supported: en-GB (code language), ro-RO

Mock text can be used in the beginning to not spend much time on translating everything

Django i18n docs: https://docs.djangoproject.com/en/3.0/topics/i18n/

[client] Build files should be placed in the ./api folder

Because some of the CSS will be shared between the client and the api, we need to have a way for the api code to be able to serve those files from within it's container.

For now we should just build the client files into the ./api/public/ folder. That folder is already in the git ignore list so it should be fine.

[api] Add tests to the buildings api

Create a api/tests/test_buildings_api.py file that tests the basic CRUD functionality of the buildings api.

We used the pytest testing framework, make sure you use of all its awesome features. :)

[DevOps] GitHub Actions pipeline for basic checks on master branch for Python

In order to make sure that the master branch is safe, we need to put in place a basic pipeline, using GitHub Actions, for running the basic scripts. The purpose of the pipeline is to ensure we don't push broken functionality in the repository.

The pipeline should make sure that the following scripts can be properly run:

python3 manage.py loaddata buildings
python3 manage.py runserver

[api] Make the pages api read-only

The create/update functionality should only be available in the django admin.
Add tests showing this is not possible from the client facing api.

[client] Make use of the API to get map data

Right now the client uses an outdated json dump of the data and the json-server package to serve it.

The client code should be able to take an environment variable with the site domain and get the data directly from the api.

You can load some dummy data into the buildings api by running the make init-db (this will also create a superuser for you in the django admin).

[DevOps] Consider configuring the app through environment variables

Rather than hard-coding things like database host, name, user, Django secret key and other things, we could take them from the environment.

This ensures that our app follows the 12-factor app design principles and makes our lives easier, because it will be easier to pass different configuration depending on the deployment.

Here is how this looks: https://testdriven.io/blog/dockerizing-django-with-postgres-gunicorn-and-nginx/

Basically, the database settings would look something like this:

DATABASES = {
    "default": {
        "ENGINE": os.environ.get("SQL_ENGINE", "django.db.backends.sqlite3"),
        "NAME": os.environ.get("SQL_DATABASE", os.path.join(BASE_DIR, "db.sqlite3")),
        "USER": os.environ.get("SQL_USER", "user"),
        "PASSWORD": os.environ.get("SQL_PASSWORD", "password"),
        "HOST": os.environ.get("SQL_HOST", "localhost"),
        "PORT": os.environ.get("SQL_PORT", "5432"),
    }
}

We could use django-configurations, which makes it easier to read from the environment. Additionally, it gives us a way to separate development and production settings.

Makefile installation of Docker fails on Ubuntu 19.10

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(shell lsb_release -cs) stable", on Ubuntu 19.10, will add the repository for "ubuntu eoan stable", but Docker.com does not provide packages for Ubuntu Eoan: docker/for-linux#833

I've worked around this by using the Docker repository for the previous version of Ubuntu:
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu disco stable"

[Map] Search locations on map based on address

As a user of the platform, I want to be able to search location on the map based on the address.

Update: The search bar is implemented in the client but there is no connection to the backend. Functionality should be implemented in Django, on the backend.

[Discussion] API and client interaction in the blog

Blog posts are generated in the same way as other pages through the Django CMS
The Django REST API should be able to return these articles
The React client should be able to add a style over them

We would need to form a better opinion on how such an architecture might look
Anyone with experience in this area is welcome to contribute to the discussion

[Django] Clean up the old unused API

Also called the "legacy API". We're at that stage where we don't have to keep legacy.

Should be safe to just completely nuke the seismic_site/api folder and remove this one line https://github.com/code4romania/seismic-risc/blob/master/seismic_site/seismic_site/urls.py#L27

While at it, the seismic_site/map_app/serializers.py is a complete duplicate (unused) of seismic_site/buildings/serializers.py (left-over from #61, my bad). This one would actually give an error if it were to be used.

[api] Add tests to the pages api

Create a api/tests/test_pages_api.py file that tests the basic view functionality of the pages api. No need to test the create/update methods as those will be removed from the client facing api.

We use the pytest testing framework, make sure you use of all its awesome features. :)

[Initialization] Dev setup data load fails

On a fresh clone

$  python3 manage.py loaddata buildings

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/db/models/options.py", line 565, in get_field
    return self.fields_map[field_name]
KeyError: 'nr_cadastral'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/django/core/serializers/json.py", line 69, in Deserializer
    yield from PythonDeserializer(objects, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/serializers/python.py", line 118, in Deserializer
    field = Model._meta.get_field(field_name)
  File "/usr/local/lib/python3.7/site-packages/django/db/models/options.py", line 567, in get_field
    raise FieldDoesNotExist("%s has no field named '%s'" % (self.object_name, field_name))
django.core.exceptions.FieldDoesNotExist: Building has no field named 'nr_cadastral'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 21, in <module>
    main()
  File "manage.py", line 17, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
    self.loaddata(fixture_labels)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/loaddata.py", line 114, in loaddata
    self.load_label(fixture_label)
  File "/usr/local/lib/python3.7/site-packages/django/core/management/commands/loaddata.py", line 172, in load_label
    for obj in objects:
  File "/usr/local/lib/python3.7/site-packages/django/core/serializers/json.py", line 73, in Deserializer
    raise DeserializationError() from exc
django.core.serializers.base.DeserializationError: Problem installing fixture '/Users/iuliux/seismic-risc/seismic_site/map_app/fixtures/buildings.json':

[api] Add tests to the blog api

Create a api/tests/test_blog_api.py file that tests the basic read functionality of the blog api.

We used the pytest testing framework, make sure you use of all its awesome features. :)

[Frontend] Create blog posts page

As a user, I want to see blog posts written on the platform, through the CMS, all in one page, so that I can inform myself about the risks of an earthquake.

The articles should support tags so that I can filter them by city.

For reference, see the third mockup image.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.