bcgov/wps-fire-perimeter

Generate fire perimeter from satellite imagery

Proof Of Concept, Generate Fire Boundaries using Google Earth Engine

About

This is a proof of concept to generate fire boundaries using Google Earth Engine.

The goal is to prove the utility of generating fire perimeter from satellite imagery, thus reducing the need for manual generation of perimeters using helicopters.

Issue: Current implementation dependant on cloud cover! Current algorithm doesn't work with lots of cloud and smoke. Smoke may cause holes in the perimiter. Consider using MODIS + VIIRS as source.

Issue: Need a government google account to productionize this.

Credit: The code that identifies fire is basead on code by https://github.com/ashlinrichardson available at: https://github.com/bcgov/wps-research/blob/master/py/gee/active_fire.js

Process / Components

flowchart LR

    subgraph Openshift
        job["Scheduled Job</br>Runs daily, generates fire perimeters for active fires > 90ha.</br>Uses publicically available data."]
        postgis[(PostGIS)]
        subgraph Servers
            pg_featurserv["pg_featurserv</br>A lightweight RESTful geospatial feature server for PostGIS."]
            rasterserv["Raster Server</br>Serve up GeoTIFF files."]
        end
    end
    WFS["https://openmaps.gov.bc.ca"]
    GEE["Google Earth Engine(GEE)"]
    s3[S3 Compliant, OCIO Object Storage Service]
    user("User with GIS software/Browser")

    job--"Query active fires > 90ha"-->WFS
    job--"Generate fire perimeter GeoTIFF"-->GEE
    job--"Store polygonized fire perimeter"-->postgis
    job--"Store fire GeoTIFF"-->s3
    pg_featurserv--"Serve features for GIS"-->postgis
    rasterserv--"Serve fire GeoTIFF"-->s3
    user--"OCG API"-->pg_featurserv
    user-->rasterserv

Demo

Proof of concept demo: https://wps-dev-featureserv.apps.silver.devops.gov.bc.ca/

pg-featureserv screenshot

GeoTIFF screenshot

General

• Create .env file (you can copy .env.example and set appropriate values)
• Assumes use of python poetry
• Google Earth Engine does not work on versions newer than 3.8.*

Ubuntu

pip install pygdal==3.0.4.10
poetry install
poetry run python -m pip install pygdal==3.0.4.10
make cli

Running

poetry run python3 -m fire_perimeter.cli

Using macports on m1

I had trouble using pyenv to install the version I need. So installing python with macports, and telling poetry to use the version I want.

sudo port selfupdate
sudo port upgrade outdated
sudo port install python38
sudo port install gdal
sudo port install proj9

Python 3.8 is installed to: /opt/local/Library/Frameworks/Python.framework/Versions/3.8

Find out where proj is installed, and set PROJ_DIR for your environment

port contents proj9

poetry env use /opt/local/Library/Frameworks/Python.framework/Versions/3.8/bin/python3
poetry run python -m pip install --upgrade pip
poetry install
poetry run python -m pip install gdal==$(gdal-config --version) --no-cache-dir

NOTE: --no-cache-dir is very important to make sure that gdal install doesn't skip numpy bindings. NOTE: order is very important here, you need to have installed numpy before gdal.

Openshift

Build image

Shortcut!

Taking some shortcuts! Skipping build in openshift, and pushing up from local

make build
docker tag wps-fire-perimeter:latest image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/wps-fire-perimeter:latest
docker login -u developer -p $(oc whoami -t) image-registry.apps.silver.devops.gov.bc.ca
docker push image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/wps-fire-perimeter:latest

Deployment

oc -n e1e498-dev process -f openshift/templates/perimeter_cronjob.yaml | oc -n e1e498-dev apply -f -

wps-fire-perimeter's Issues

Add missing topics

TL;DR

Topics greatly improve the discoverability of repos; please add the short code from the table below to the topics of your repo so that ministries can use GitHub's search to find out what repos belong to them and other visitors can find useful content (and reuse it!).

Why Topic

In short order we'll add our 800th repo. This large number clearly demonstrates the success of using GitHub and our Open Source initiative. This huge success means it's critical that we work to make our content as discoverable as possible. Through discoverability, we promote code reuse across a large decentralized organization like the Government of British Columbia as well as allow ministries to find the repos they own.

What to do

Below is a table of abbreviation a.k.a short codes for each ministry; they're the ones used in all @gov.bc.ca email addresses. Please add the short codes of the ministry or organization that "owns" this repo as a topic.

That's it, you're done!!!

How to use

Once topics are added, you can use them in GitHub's search. For example, enter something like org:bcgov topic:citz to find all the repos that belong to Citizens' Services. You can refine this search by adding key words specific to a subject you're interested in. To learn more about searching through repos check out GitHub's doc on searching.

Pro Tip 🤓

• If your org is not in the list below, or the table contains errors, please create an issue here.

• While you're doing this, add additional topics that would help someone searching for "something". These can be the language used javascript or R; something like opendata or data for data only repos; or any other key words that are useful.

• Add a meaningful description to your repo. This is hugely valuable to people looking through our repositories.

• If your application is live, add the production URL.

Ministry Short Codes

Short Code	Organization Name
AEST	Advanced Education, Skills & Training
AGRI	Agriculture
ALC	Agriculture Land Commission
AG	Attorney General
MCF	Children & Family Development
CITZ	Citizens' Services
DBC	Destination BC
EMBC	Emergency Management BC
EAO	Environmental Assessment Office
EDUC	Education
EMPR	Energy, Mines & Petroleum Resources
ENV	Environment & Climate Change Strategy
FIN	Finance
FLNR	Forests, Lands, Natural Resource Operations & Rural Development
HLTH	Health
IRR	Indigenous Relations & Reconciliation
JEDC	Jobs, Economic Development & Competitiveness
LBR	Labour Policy & Legislation
LDB	BC Liquor Distribution Branch
MMHA	Mental Health & Addictions
MAH	Municipal Affairs & Housing
BCPC	Pension Corporation
PSA	Public Service Agency
PSSG	Public Safety and Solicitor General
SDPR	Social Development & Poverty Reduction
TCA	Tourism, Arts & Culture
TRAN	Transportation & Infrastructure

NOTE See an error or omission? Please create an issue here to get it remedied.

It's Been a While Since This Repository has Been Updated

This issue is a kind reminder that your repository has been inactive for 180 days. Some repositories are maintained in accordance with business requirements that infrequently change thus appearing inactive, and some repositories are inactive because they are unmaintained.

To help differentiate products that are unmaintained from products that do not require frequent maintenance, repomountie will open an issue whenever a repository has not been updated in 180 days.

• If this product is being actively maintained, please close this issue.
• If this repository isn't being actively maintained anymore, please archive this repository. Also, for bonus points, please add a dormant or retired life cycle badge.

Thank you for your help ensuring effective governance of our open-source ecosystem!

Lets use common phrasing

TL;DR 🏎️

Teams are encouraged to favour modern inclusive phrasing both in their communication as well as in any source checked into their repositories. You'll find a table at the end of this text with preferred phrasing to socialize with your team.

Words Matter

We're aligning our development community to favour inclusive phrasing for common technical expressions. There is a table below that outlines the phrases that are being retired along with the preferred alternatives.

During your team scrum, technical meetings, documentation, the code you write, etc. use the inclusive phrasing from the table below. That's it - it really is that easy.

For the curious mind, the Public Service Agency (PSA) has published a guide describing how Words Matter in our daily communication. Its an insightful read and a good reminder to be curious and open minded.

What about the master branch?

The word "master" is not inherently bad or non-inclusive. For example people get a masters degree; become a master of their craft; or master a skill. It's generally when the word "master" is used along side the word "slave" that it becomes non-inclusive.

Some teams choose to use the word main for the default branch of a repo as opposed to the more commonly used master branch. While it's not required or recommended, your team is empowered to do what works for them. If you do rename the master branch consider using main so that we have consistency among the repos within our organization.

Preferred Phrasing

Non-Inclusive		Inclusive
Whitelist	=>	Allowlist
Blacklist	=>	Denylist
Master / Slave	=>	Leader / Follower; Primary / Standby; etc
Grandfathered	=>	Legacy status
Sanity check	=>	Quick check; Confidence check; etc
Dummy value	=>	Placeholder value; Sample value; etc

Pro Tip 🤓

This list is not comprehensive. If you're aware of other outdated nomenclature please create an issue (PR preferred) with your suggestion.

Add project lifecycle badge

No Project Lifecycle Badge found in your readme!

Hello! I scanned your readme and could not find a project lifecycle badge. A project lifecycle badge will provide contributors to your project as well as other stakeholders (platform services, executive) insight into the lifecycle of your repository.

What is a Project Lifecycle Badge?

It is a simple image that neatly describes your project's stage in its lifecycle. More information can be found in the project lifecycle badges documentation.

What do I need to do?

I suggest you make a PR into your README.md and add a project lifecycle badge near the top where it is easy for your users to pick it up :). Once it is merged feel free to close this issue. I will not open up a new one :)