ucl-eo/workshop2022

Using Earth observation for Crop Monitoring (iEOCM) Workshop

Workshop2022

Online practicals:

Workshop information webpage

Installation

1. Install Miniconda: https://docs.conda.io/en/latest/miniconda.html

2. Download the repository:

git clone https://github.com/UCL-EO/Workshop2022.git

3. Install packages:

cd Workshop2022/
conda env create -n uclnceo --force -f environment.yml

4. Enable extensions

#!/bin/bash

# activate uclnceo envrionment
conda activate uclnceo

jupyter serverextension enable --py nbgitpuller --sys-prefix
jupyter contrib nbextension install --user
jupyter nbextension enable --py widgetsnbextension
jupyter nbextension enable python-markdown/main

# Install a JupyterLab extension for demonstration purposes
jupyter labextension install @jupyterlab/geojson-extension 
jupyter labextension install @jupyter-widgets/jupyterlab-manager
jupyter labextension install jupyter-leaflet
jupyter trust notebooks/*.ipynb *.ipynb

# kernel setup
python -m ipykernel install --name=conda-env-uclnceo  --display-name 'conda env:uclnceo' --user

5. Run jupyter-lab and load the notebooks.

Welcome to the Using Earth observation for Crop Monitoring (iEOCM) Workshop! You will find the following content:

Tamale Workshop

MORNING SESSION [09:00-11:00 2 hours]

• Welcome: GSSTI Director [5 mins]
• Welcome: Structure of the workshop: Kofi Asare GSSTI [5 mins]
• Perspectives and needs: MOFA regional director [20 mins]
• Cropping issues in North of Ghana: ADRA Kankam Boadu [20 mins]
• Overview and achievements: Prof Lewis [20 mins]
• Role of ground data and summary findings from 2021: Kofi Asare GSSTI [20 mins]
• Weather and yield in-season forecast: Nicola Pounder, Assimila [20 mins]
• Discussion and future priorities [GSSTI to chair] [15 mins]

Accra Workshop

MORNING SESSION [09:00-11:00 2 hours]

• Welcome: GSSTI Director [5 mins]
• Welcome: Structure of the workshop: Kofi Asare GSSTI [5 mins]
• Perspectives and needs: MOFA yield estimation director [20 mins]
• Overview and achievements: Prof Lewis [20 mins]
• Role of ground data and summary findings from 2021: Kofi Asare GSSTI [20 mins]
• Climate and yield: Dilys MacCarthy [20 mins]
• Weather and yield in-season forecast: Nicola Pounder, Assimila [20 mins]
• Discussion and future priorities [GSSTI to chair] [15 mins]

Practical Session [11:20:12:40] [1:40-2:00]

Groups of 5, run each session x 4

Practicals:

• You may want to register for a GEE account before starting the practicals.

• [DEA showcase]: DEA Edward Boamah
• Biophysical binder (Feng): EO & field data explorer
• ERA5 Weather data (Nicola) and Rain Watch (Kofi)
• Model data assimilation (Jose/Hongyuan): Crop model sliders, DA-Me, yield context plot
• Maize mapper GEE visualisation and MOFA yield (Qingling)
• [Field measurements demo] Kenneth

Workshop technical presentations 2:30-4:00

(15 mins)

• DEA classification in Ghana (Digital Earth Africa)
• Classification (China Agricultural University)
• Crop Modelling & Ensembles:(Hongyuan)
• Crop model video:(Hongyuan)
• EO interpretation (talk on issues and methods) (feng)
• Introduction to data assimilation (Jose/Hongyuan/CAU)
• Ensemble Crop DA (jose)
• Discussion and Reflections: GSSTI (Kofi to lead)

Layout

• data: folder for data. Keep small e.g. CSV files etc here. If you need larger data, download via postBuild script. carto subfolder has maps of Ghana.
• figures: Figures, logos, ....
• notebooks: Where notebooks go
• presentations: pdf version of presentations (needed?)
• wkshp_codes: python code to be used by notebooks

workshop2022's Issues

Maize mapper and explorer

• decide visualisation scheme by Monday
• QA for MODIS LAI time series plotting
• Aspatial data explorer
• Spatial VI (link with DigitalAfrica team?)
• Instruct GEE account approval with @MarcYin

Priorities:

• legend/palette issue
• apply-mask as default
• axis labels (year, lai, precp, && their units) for TS figures
• upload mofa csv file(s) to GitHub
• add scatterplots as subplots: mofa yield vs precp
• displaying district/county names on click

Targets set on 11-Mar-2022:

• generate histogram per QA level
• generate IO webpage
• Tidy up, fix all bugs
• change rain m to mm
• add 1-to-1 line for scatterplot

Additional feature only if have time:

• all districts, instead of northern (in progress)
• some demo of toolboxes : e.g. upload own vector / raster
• user-defined AOI
• drawing tools: TBD: remove?
• on-click: per-pixel TS explorer
• graph of ALL properties of pixel, on click
• yield estimate to be related: e.g average yield over the masked area
• add scatterplots as subplots: mofa yield vs empirical yield; and show stats of graph

WOFOST demo needs to be run for other places (possibly other crop?)

Animation over Tamale seen with S2 growing season 2021

README organisation

Each talk or interactive thing to have its own section with

• A link to the material (github)
• A list of issues (on github)
• A link to binder (if needed)
  Need to check the syntax of the link to Binder

Biophysical parameters notebook

• 1. Planet LAI 01-Satellite_data.ipynb
• 2. GEE backend NDVI/LAI 01B_NDVI_LAI.ipynb [End of Monday]
• 3. Require WOFOST ensemble over the field measurement sites from @jgomezdans and @Ma-hy [End of Monday]
• 4. Integrate WOFOST slider into GEE backend LAI [End of Wednsday]

Function to generate ensemble

Fengs mapper issues

@jgomezdans @MarcYin

When do you assume harvesting is? I can see field notes mention ‘ready for harvesting’ but don’t seem to say when that was. Surely we should constraint or at least check that we have that right.

The autofit doesn’t seem to be working – is that broken ATM?

Im playing with Fengs maize mapper, and it would be really useful if GS 1 and 2 were indicated on the plots … I can see you can read off when 1, 2.0 GS is, but that’s quite hard to do. Esp since 1 isnt shown directly. Feng - Can you pls show GS 1 and GS 2 as vertical lines in both the modelling plots and the data plots?

Also, once you go into LAI movie mode, there doesn’t seem a way to switch back to showing yield map without reloading, which is clunky. Is that fixable?

Also, whilst there is ‘help’ for the terms used on the sliders, there is no help for the quantities plotted … also, no units given for these (that could be in the help). It is hard to work out what you are seeing.

Also, I cant see the measured yield data anywhere in the plots - where is that?

instructions for installing on own computer

@MarcYin can you put some instructions on the readme for a local install for users

That might be
get miniconda (point them at site)
cd $DISTO
conda env create -f environment.yml -n uclnceo
postBuild

or whatever

Add year 2021 to maize mapper

Plot water use effiiciency in wofost outputs

Crop modelling presentation

Tasks

• 2000-2021 Ensemble Generation
• pixel-by-pixel DA
• Fewer parameters ensemble generation

Keynote Content

• Crop Model principles
• WOFOST introduction
• WOFOST implementation and ensemble generation
• Multi-years Yield Map
• In season simulation

Ensemble meteo & modelling

• @NPounder to produce a meteo file with seasonal foracasts until 1st-2nd week Sept 2021 over study area (~lat=9.5, lon=-0.5)
• @jgomezdans to run an ensemble (parameters+meteo) over growing season 2021
• Have a notebook that plots the meteo data and allows selection of a field to do assimilation until mid Sept and forecast end of season

• Only single location as limited time etc.

DA notebook

• Model exploration with parameters ("sliders")
• Automatically fit the LAI ("DA button")
• Plot yield distribution for all fields w/boxplots
• Per pixel LAI exploration
  • Show yield
  • Show parameter spatial distribution

Load LAI & yield into wofost demonstrator

Show how you can manually tune th emodel to match observations

DA presentation

• Contact HangHui re MCMC
• Need to get some form of within-season forecast example from Hongyuan/Nicola

Talk contents

• DA concepts (just an outline, mostly pictures)
  • Drivers/States/Parameters and models
  • Model & observation uncertainties (examples)
• Assimilation: blending models & observations. Mostly pictures and no maths
  • Correct the model trajectory/state: filters
  • Pros/cons
  • Variational methods: Parameter estimation
  • Pros/cons
  • Ensemble methods
• Example of system working with the crop model
  • Ensemble generation (parameters, possibly weather)
  • Linking model predictions with observed LAI
  • Yield prediction
  • Model parameter interpretation
• DA and within-season forecasting introduction @Ma-hy to provide some slides

wofost running exercises

• What do you have to do to double yield?
• What do you hav to do to do a short vs long season simulation?
• Fit the model to the observations

wofost sliders should show the default model values

Sort out field pictures by field in Google Photos

WE need a URL for each field folder photos

@MarcYin

Digital Earth Africa practical link

Need a link from the DEA practical

Travel and hotels - to contact Clarity

• international flights between London and Accra
• domestic flights between Accra and Tamale
• Tamale hotel, Mar 20-23 Can @asarefi please confirm if Tamale hotel can be booked and invoiced by GSSTI?
• Accra hotel, Mar 19
• Accra hotel, Mar 23-26
• need passport info from @jgomezdans @profLewis @MarcYin to book tickets: passport number, expiry date, date of birth, mobile number, and ID email address.
• Risk assessment
• travel insurance
• book in-person visa appointments

Model runs fixes

• Model runs need reduced parameter sets
• AMAX scalar anchored using AMAXTB_200
• Plot some parameter trajectories in max_LAI space and/or length of season space
• Ensure same wofost model is run in here & jasmin

bug in Satellite script

Ive tried to run Fengs script standalone, with an environment from the yml file.

I get the error below. So, you need the retry package, which Ive put in the environment file. And will retry....

Platform not recognized, using system temp directory for PCSE settings.
Platform not recognized, using system temp directory for PCSE settings.
Building PCSE demo database at: /var/folders/qk/t8kmgc253rg566r5qy3wrq9w0000gn/T/.pcse/pcse.db ... OK

ModuleNotFoundError Traceback (most recent call last)
File ~/Documents/GitHub/Workshop2022/notebooks/create_map.py:21, in
19 from map_utils import get_lai_gif, get_pixel, get_field_bounds, da_pix, get_lai_color_bar
20 from map_utils import debounce
---> 21 from wofost_utils import create_ensemble, wofost_parameter_sweep_func, get_era5_gee
22 from wofost_utils import ensemble_assimilation
24 from ipywidgets import Image as ImageWidget

File ~/Documents/GitHub/Workshop2022/notebooks/./python/wofost_utils.py:35, in
33 import pip
34 import shutil
---> 35 from retry import retry
37 try:
38 ee.Initialize(opt_url="https://earthengine-highvolume.googleapis.com/")

ModuleNotFoundError: No module named 'retry'

EO interpretation talk

Framework of presentation: [By Friday morning]

1. Basic concept of monitoring vegetation with remote sensing
   1.1 plant canopy scale visualisation
   1.2 leaf level with prospect
   1.B NDVI to LAI and why not
2. Archetypes over the US and some validation
3. Modelling and application
4. How to Ghana fields and issues
5. Results and validation against the ground measurements
6. Emphasis the importance of sentinel 2 and planet on monitoring smallholder farming and why we do the archetypes

Create github pages

https://docs.github.com/en/pages/quickstart

Send invitations to potential participants

Qingling and Kofi has updated the draft invitation. Any final proof @profLewis ?
https://liveuclac-my.sharepoint.com/:w:/r/personal/ucfaqqw_ucl_ac_uk/_layouts/15/Doc.aspx?sourcedoc=%7B6D15C7A1-C4FF-4E2B-A8A1-C1D6120642DC%7D&file=Stakeholder-invitation-letter-draft.docx&action=default&mobileredirect=true