sustainlab-group/africa_poverty

africa_poverty's Issues

Website Proof of Concept

Hi SustainLab Group.

I found this project very interesting and would like to contribute to expanding it. I would like to create a website that could use this model to show poverty on a large scale and also provides trends and analytics for poverty over time. Planet allows daily access to satellite data which would allow me to get high frequency insights. https://www.planet.com/products/monitoring/

There are websites like https://sedac.ciesin.columbia.edu/data/collection/povmap/maps/gallery/search?facets=theme:poverty&facets=region:africa but nothing that seems to give an easy to use UI which allows someone to shift a map with some simple scrolling and clicking. An abstract example to something I would like to make is the John Hopkins Website Coronavirus Website (https://coronavirus.jhu.edu/map.html).

Some research has been published on the possibility of this in https://www.researchgate.net/publication/344038389_High-Resolution_Poverty_Maps_in_Sub-Saharan_Africa and, of course, https://www.nature.com/articles/s41467-020-16185-w .

Do you all believe that a project and website like this would be impactful and useful for the real world?

training step

Hi,

I've successfully run the first steps (0_export_tfrecords.ipynb, 1_process_tfrecords.ipynb, 2_create_incountry_folds.ipynb, 3_download_gadm_shapefiles.sh), but it looks like there's a gap in the training step.??

Thank you in advances for your help.

preprocessing scripts incomplete?

I've successfully run the first step to download the raw TFRecords from Earth Engine, but it looks like there's a gap between the first download script and the preprocessing. My filenames are all named according to a single country, single year, and chunk number, but it looks like process_tf_records_dhs.ipynb expects combined files for a single country with the name format lx_median_{year_range}_{country}_dhslocs_ee_export.tfrecord.gz. I don't see anywhere in the repo to create these; is there a missing preprocessing step?

data preprocessing

Hello, I am trying to run the project but i have encountered several issues Especially in preprocessing part.

After finish all the steps downloading part and export the Earth Engine data to google cloud storage i go to process_tfrecords notebook the main issue here is my exported earth engine file names in format like this: {country_name}{year_range}.tfrecord.gz
But in notebook process_tfrecords_dhs.ipynb name should be in this type : /lx_median{year_range}_{country}_dhslocs_ee_export.tfrecord.gz

I have change the name format and moved on but last part (Process TFRecords) none of the run processing functions are working i am getting error like:

• list index out of range
• There is no such file:

for instance angola's data stored as angola2011_xx.tfrecord.gz to angola2015_xx.tfrecord.gz in cloud storage. But notebook tries to find angola2009-11.tfrecord.gz

• Cluster index not foud in tfds file: in REQUEIRED_KEYS list there is "cluster index" but some of my tfds files not inclues this.

I couldn't figure out where is the mistake or did i miss a step to create lx_median_{year_range}_{country}_dhslocs_ee_export.tfrecord.gz Can you please explain and help about this issue ?
Thanks

Edit:
I am inspecting the code most probably issues happens due to lacking of cluster indexes in tfrecord files.
And maybe i should concatenate the tfrecords files.

license and reproducibility

I realised license is not specifically mentioned in this project. I am running a similar project where I would need to directly re-use some of the code scripts as exactly as they are, especially the data pre-processing part. I would like to understand if there something I can know regarding the permission to use the code scripts and citing.