ahmetkucuk/query-solar-database

Java project for creating PostgreSql based solar database.

query-solar-database's Issues

Validation of HEK Data

We need to validate each event record downloaded from HEK. Looking at Mike's Dataset paper and LSDO (see http://lsdo.dmlab.cs.gsu.edu/ some data quality check can be found here: https://github.com/ahmetkucuk/lsdo-generator), If we receive any bad event record (that could be bad reporting module, bad polygon etc.) we should not save it into database.

@berkay-aydin Do you mind elaborating this task by specifying exact requirements for us to run tracking algorithm without having problem with data?

Create ISD Web Container

This project should be another container inside pipeline: https://github.com/ahmetkucuk/IntegratedSolarDatabase

Before starting creating container, you need to first run IntegratedSolarDatabase in your local and connect it to our remote database. See README for running it locally: https://github.com/ahmetkucuk/IntegratedSolarDatabase/tree/hamdi

Before you run it (bee run), I will send you a app.conf file that has credentials to our database in GSU servers. You need to add this file under /conf file (we cannot push it to github bec. One can connect to our servers). Example: https://github.com/ahmetkucuk/IntegratedSolarDatabase/tree/hamdi/conf

1. Create folder named isd under ./deploy
2. Create a Dockerfile
3. Inside Dockerfile, you need to do following.

• Get golang image and install beego framwork:

FROM golang:1.8

# Install beego & bee
RUN go get github.com/astaxie/beego
RUN go get github.com/beego/bee

• Copy IntegratedSolarDatabase into appropriate folder and run beego:

cd ./CORRECT_DIR

RUN git clone https://github.com/ahmetkucuk/IntegratedSolarDatabase.git

bee run

4. Once you finished, you will need to write new ./conf/app.conf, so that it connects to locally running postgresql container instead of remote server or any other database. Note that, it will not able to find stored procedure on locally running postgresql bec. we are not automatically adding stored procedure. I will create another task that needs to be done before this.

See tutorial about how to create beego image for more detail: https://github.com/lei-cao/beego-in-action/blob/master/en/beego-in-docker.md

I will try to add more detail but it would be better if you ask questions so that I know what you are missing. This is quite big task, don't panic.