jiaxuanyou/crop_yield_prediction

Crop Yield Prediction with Deep Learning

Crop yield Prediction with Deep Learning

The necessary code for our paper, Deep Gaussian Process for Crop Yield Prediction Based on Remote Sensing Data, AAAI 2017 (Best Student Paper Award in Computational Sustainability Track). We are glad to win the "Best Big Data Solution" in World Bank Big Data Innovation Chanllenge as well.

Here is a brief introduction on the utilities for each folder.

• "/1 download data" How we download data from Google Earth Engine to Google Drive. Users then need to export data from Google Drive to their local folder, e.g., their clusters. The trick there is that we first concatenated all images across all the years available (say 2003 to 2015), then download the huge image at once, which could be hundreds of times faster.
• "/2 clean data" How the raw data is preprocessed, including slicing the huge images to get individual images, 3-D histogram calculations, etc.
• "/3 model" The CNN/LSTM model structure, written in tensorflow (v0.9). The Gaussian Process model, written in python.
• "/4 model_batch" Since we are training different models for each year and each month, a batch code is used for training.
• "/5 model_semi_supervised" A recent contribution, extending the model with semi-supervised deep generative model, however it doesn't work well. We are happy to discuss the model if you can make it work.
• "/6 result_analysis" Plot results, plot yield map, etc.

For more information, please contact Jiaxuan You.

[email protected].

crop_yield_prediction's Issues

Links to paper broken

The links to the paper in the readme.md are broken.

Download the data

Your first folder shows the download data, but how to download the data

can't read the paper

I can't read the paper, can I ask if someone solved it?

EEException: Collection.loadTable: Collection asset 'ft:1S4EB6319wWW2sWQDPhDvmSBIVrD3iEmCLYB7nMM' not found.

def export_oneimage(img,folder,name,region,scale,crs):
  task = ee.batch.Export.image(img, name, {
      'driveFolder':folder,
      'driveFileNamePrefix':name,
      'region': region,
      'scale':scale,
      'crs':crs
  })
  task.start()
  while task.status()['state'] == 'RUNNING':
    print('Running...')
    # Perhaps task.cancel() at some point.
    time.sleep(10)
  print('Done.', task.status())

county_region = ee.FeatureCollection('ft:1S4EB6319wWW2sWQDPhDvmSBIVrD3iEmCLYB7nMM')

export_oneimage(img.clip(region), path, fname, region, scale, crs)


---------------------------------------------------------------------------
HttpError                                 Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries)
    333   try:
--> 334     return call.execute(num_retries=num_retries)
    335   except googleapiclient.errors.HttpError as e:

7 frames
HttpError: <HttpError 400 when requesting https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/image:export?alt=json returned "Collection.loadTable: Collection asset 'ft:1S4EB6319wWW2sWQDPhDvmSBIVrD3iEmCLYB7nMM' not found.". Details: "Collection.loadTable: Collection asset 'ft:1S4EB6319wWW2sWQDPhDvmSBIVrD3iEmCLYB7nMM' not found.">

During handling of the above exception, another exception occurred:

EEException                               Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/ee/data.py in _execute_cloud_call(call, num_retries)
    334     return call.execute(num_retries=num_retries)
    335   except googleapiclient.errors.HttpError as e:
--> 336     raise _translate_cloud_exception(e)
    337 
    338 

EEException: Collection.loadTable: Collection asset 'ft:1S4EB6319wWW2sWQDPhDvmSBIVrD3iEmCLYB7nMM' not found.

Missing files

acres_harvested.csv file is referenced in GP_crop_v3.py, but it's not in the repo.

Sample content of Locations.csv file is missing

can you please add sample data for locations data

Incomplete readme

• Please add instructions for how to run the code, the most suitable python version, a requirements.txt file and other software requirements.

• A master main function which calls all the latest necessary functions based on user's need can make this project more readable.

Administrative shapefiles

Hello, Could tell me where I can download the administrative shapefiles of USA in the codes? I downloaded shapefiles what can not match your /.cvs files. Some county_num I can not get. For example, I don't the STATE_num.COUNTY_num:1.119.

Can't access the paper for the project

All links seem dead. Please advise.
Thanks!

Data Missing

Can you suggest me an alternate, now that the fusion tables are not avaiable, and also the necessary changes ?