kakada/event_base_surveillance

Run the example

Generate the certificates (only needed once):

docker-compose -f elastic/certificates/create-certs.yml run --rm create_certs

Start two Elasticsearch nodes configured for SSL/TLS:

docker-compose up -d

Access the Elasticsearch API over SSL/TLS using the bootstrapped password:

docker run --rm -v es_certs:/certs --network=es_default elasticsearch:7.4.1 curl --cacert /certs/ca/ca.crt -u elastic:PleaseChangeMe https://es01:9200

The elasticsearch-setup-passwords tool can also be used to generate random passwords for all users: Windows users not running PowerShell will need to remove \ and join lines in the snippet below.

docker exec es01 /bin/bash -c "bin/elasticsearch-setup-passwords \
auto --batch \
-Expack.security.http.ssl.certificate=certificates/es01/es01.crt \
-Expack.security.http.ssl.certificate_authorities=certificates/ca/ca.crt \
-Expack.security.http.ssl.key=certificates/es01/es01.key \
--url https://localhost:9200"

Tear everything downedit

To remove all the Docker resources created by the example, issue:

docker-compose down -v

For more detail on how to configure docker elasticsearch with ssl/tls, please visit https://www.elastic.co/guide/en/elasticsearch/reference/7.4/configuring-tls-docker.html

event_base_surveillance's Issues

elasticsearch and kibana

Notify 3rd-parties - Webhook

As a program admin
I want to be able to configure the notification to 3rd-party webhook via basic authentication or api token

[Webhook] Program 3rd-party management

Allow program admin to manage 3rd-party webhook(s)

• List 3rd-parties
• Create, update, activate/disactivate 3rd-party
  with authentication method(http base, api token)

Allow Sidekiq web route to access only system admin user

Event - filter UI for program user depends on field and value

Add current milestone to event model

Add current milestone in Event model to support funnel metric analytic in elasticsearch

Dashboard - Event Tracking Timeline

Building a statistic dashboard of Event tracking timeline for each Event detail (UI driven)

Adding map to event detail by using openstreetmap

display no location database if that place no geodatabase

activity log

Event type management

System admin design built-in share event type
where event type can be adding more by program admin
and can share to the rest of program

[Telegram] Alert threshold - template management

Allow program admin to create, update, activate/deactivate
template message which allowed to embed pre-defined variable(s) on a specific event type on each event step(status)

in e.g: {{event.status}}, {{event.reporter}}, {{event.reported_date}}, {{event.type}}, {{event.location}}, ..etc.

Template designer can select/typing variable to embed into the template

[Webhook] Alert threshold - notify 3rd-party

Allow user to configure push notification to 3rd-party(can select from 3rd-parties program or manual configure) for a specific alert

Map visualization

visualize program/model event type as a cluster with color classify as assigned in the event type

Dashboard - Event Statistic Report

Dashboard of

1. Total # of each status(NEW, INVESTIGATING, ..etc.)all event type
2. Total # of each status of each event type
3. #average of time(day) consuming on each status
4. #average response time(day) from New to Confirmed/Rejected

with filterability:

1. date range: last week, month, quarter, year + custom range
2. Event type, location, ..etc.

Elasticsearch: fix event milestone schema

Enhance milestone

• add drag-drop to milestone list
• hide source field from milestone

Allow user to upload more than one reference file for file field

Role Management

As a system admin
When I logged in
Then it should allow me to manage role(list, create, update, delete(mark as disabled)

event history tracker

switch language

• Languages such as en and km

Program Role Permission Management

Allow program admin to create new role with
assigned built-in permission

1. Event - ready-only/update/delete
2. Report - none/read-only/update/delete

Notification: integration with telegramp Api and allow design message template

Authentication using devise with role

Elasticsearch event model structure

Event model structure in an example:

{
event_type: {
id: 1,
name: 'Zika'
},
location: {
province: {
id: '01',
name: 'Banteay Meanchey'
},
district: {
id: '0101',
name: 'Munkol Borey'
},
commune: {
id: '010101',
name: 'Banteay Neang'
},
village: {
id: '01010101',
name: 'Ou Thom'
}
},
milestones: {
verification: {
field_1: 'abc',
conduct_date: 20190101,
...
},
risk_assessment: {
conduct_date: 20190103,
field_1: 'high',
...
},
....
}
..etc.
}

[for hub connector] Rest API web service to push & pull data of the model/program

Event: update to use Uuid as primary key

Sharing event display detail event-form incorrect

Create 2 programs (CDC & GDAPH):

1. Login as gdaph program admin -> shared an event type(H1N1) to CDC
2. created a new H1N1 event
3. Log out
4. Log in as CDC program admin
5. View detail of H1N1 which is shared by GDAPH -> CDC form is rendering -> GDAPH form with detail info should be rendered

Elasticsearch: generate index classify by each program. ex: 'events_cdc'

Allow Dashboard route to access only system admin & program admin user

Authentication: add Mandrill email

https://mandrillapp.com/api/docs/index.ruby.html

Write/update/destroy event to Elasticsearch as Async(Sidekiq queued job)

As the system
When user create/update/destroy event
Then it should add to queue job in sidekiq to let it run asynchronously(background)
So it won't affect the user performance

Map location and point for rendering in map

Map(Leaflet) Integration

Reference: https://leafletjs.com/

Elasticsearch: Event shared to program

Milestone: error mapping field

Mapping with select one(risk level) UI error

Step to reproduce:

1. I have 2 milestones(New and Verification)
2. Create a new milestone Risk Assessment
3. Create a risk level level as mapping field -> map to risk level of New milestone
4. Save milestone
5. Create a new event -> complete form of New & Verification -> then click on edit, enter form Risk Assessment -> raise error as the attached screenshot

When visiting milestone form design -> then risk level mapping to Number of Case field but not Risk Level as the first saved.

Elasticsearch: update event schema

Design form to support dynamic program event model

Elasticsearch recreate index in case somethings went not well in Elasticasearch

As a developer
I want to be able to run a recreate elasticsearch indexing task When there is s.th went not well or error happened on the system which linked Rails & Elasticsearch database didn't sync

Enhance event UI

• add required field
• update date picker

User Role Permission

Allow system admin to create new admin
to program

Event model structure

Design model structure to following

1. Event - New -> Investigation -> Confirmed/Rejected -> Closed

Each step has its own different attributes/fields, it should support to report statistic structure

Refactor model name

Authentication: add button showing confirmation url

resend the confirmation to user email and show confirmation link for copying

Implement translation

Friendly event uuid

make it short and readable, and it should use for a 3rd-party app to send an update request with this friendly UUID

Program Management

As a system admin
When I logged in to the system
Then it should allow me to manage program(list, create, update, delete(mark as disabled)

event detail: add close event

1. after close event cannot be touched

3rd-party client app management

Allow program admin to manage client app authorization
-list, create with randomizing access_token(OR client_id, client_secret), whitelist IP(s), activate/deactivate)

with whitelist attributes allowed for pull(field a, b, ..) & push(field x,y,z, ..)

Authentication

Allow user to login by using devise
categorize user following the program and role
(CDC - admin/user, GDAPH - admin/user)