gee-community/gee_tools

A collection of tools to work with Google Earth Engine Python API

geetools

Google Earth Engine tools

Google Earth Engine is a cloud-based service for geospatial processing of vector and raster data. The Earth Engine platform has a JavaScript and a Python API with different methods to process geospatial objects.

The geetools package extends the Google Earth Engine Python API with pre-processing and processing tools for the most used satellite platforms by adding utility methods for different Earth Engine Objects that are friendly with the Python method chaining using the geetools namespace.

There is JavaScript module that you can import from the code editor that has similar functions (not exactly the same) and it's available here.

Installation

pip install geetools

Why using it ?

New utility methods and constructors are added to most of the GEE classes. They can be simple wrapper for repetitive tasks, complex algorithm or mandatory preprocessing steps. The goal is to make the code more fluid and easy to read for researchers, students and analysts.

The package design is mostly performing server-side computation making it also very friendly with commercial users of Earth Engine.

This small example wrapping of the excellent ee_extra package functionalities shows how to preprocess sentinel 2 data in 5 lines of code:

import ee
import geetools #noqa: F401
import pygaul # another gee-community package to access FAO GAUl 2015 dataset

# we assume you are already authenticated to GEE
ee.Initialize.geetools.from_account("toto") # yes we also support multi users

amazonas = pygaul.Items(name="Amazonas").centroid()

S2 = (
   ee.ImageCollection('COPERNICUS/S2_SR')
   .filterBounds(point)
   .geetools.closest('2020-10-15') # Extended (pre-processing)
   .geetools.maskClouds(prob = 70) # Extended (pre-processing)
   .geetools.scaleAndOffset() # Extended (pre-processing)
   .geetools.spectralIndices(['NDVI','NDWI','BAIS2'])) # Extended (processing)

gee_tools's Issues

Opacity not working in maptool.py

In geetools/ui/maptool.py the option to add opacity for addLayer does not seem to work. I think it is because in row 103 of that script you did not include opacity=params['opacity'] when creating the folium.TileLayer ....

GEE Python client -- add date to timestamp, filter images by coverage?

Hi Rodrigo -
(You helped me get gee_tools set up a few years ago! Thank you!)

I'm trying to make and download cloud-free timelapse videos of specific regions in Greenland, with the image date on each scene. I "think" I have it going in the code editor (ugly, I know!)
https://code.earthengine.google.com/?accept_repo=users/leighastearns/shared

I'd like to get it working in Python, since that's a lot easier and faster. I can get a basic video downloaded with batch.Export.video.to.drive, but my questions:

1. How do I get the date stamp in the python version?
2. Is there a way to filter out scenes with only partial coverage of my ROI (there are a number that only cover a small corner of the region)?
3. Ultimately, I'd like to combine L8_SR and Sentinel 2 scenes into a new ImageCollection, so I can densify my temporal spacing. Can I do that using "inner join" in the python version too?

These seem like they should be such easy tasks, but I seem to be going in circles parsing them.

thanks for any help,
leigh

Composite module missing

Hi there!

I am trying the notebooks attached with the master release. Just upgraded geetools using pip but I keep getting a:

ImportError: cannot import name 'composite'

I can import all other modules fine. I saw that the module composite was added 3 days ago, maybe it isn't yet available on pip?

Thanks!

name pattern issue

Hi! Used to work in the past, but when I batch export sentinel imagery with only '{:system_date:}' as the name pattern the imagery all exports as ":system_date:" rather than its respective name e.g. "2017-01-12"

Copernicus/S2 imagery not working

hey @fitoprincipe I'm working with geetools in jupyter, but I've got some problems with visualising Copernicus Sentinel 2 imagery. It works well with Landsat. I also noticed I cannot run ee.mapclient, it gives me this error:

Exception in thread Thread-4: Traceback (most recent call last): File "/home/maykef/anaconda3/lib/python3.7/threading.py", line 917, in _bootstrap_inner self.run() File "/home/maykef/anaconda3/lib/python3.7/site-packages/ee/mapclient.py", line 125, in run self.parent = Tkinter.Tk() File "/home/maykef/anaconda3/lib/python3.7/tkinter/__init__.py", line 2023, in __init__ self.tk = _tkinter.create(screenName, baseName, className, interactive, wantobjects, useTk, sync, use) _tkinter.TclError: no display name and no $DISPLAY environment variable

gee_tools: Interactive Maps with Two Options

Option 1: maptool

I followed the code in your notebook.

col = ee.ImageCollection('COPERNICUS/S2')
site = ee.Geometry.Point([-72,-42])
col = col.filterBounds(site).filterMetadata('CLOUD_COVERAGE_ASSESSMENT','less_than',40)
image =ee.Image(col.first()) 

geom = image.geometry()
visParam = {'bands':['B8','B11','B4'], 'min':0,'max':5000}
# addLayer has a parameter called inspect
inspect = {'data':image,'reducer':'mean'}
from geetools import maptool
Map = maptool.Map()
Map.addLayer(image,visParam,'Sentinel 2')
Map.centerObject(image)
Map.addLayer(geom, name = 'Image Boundary',inspect = inspect)
Map.show()

AttributeError: 'dict' object has no attribute 'iteritems'

Option 2: ipymap

from geetools import ipymap
Map2 = ipymap.Map()
Map2.show()

C:\ProgramData\Anaconda3\lib\site-packages\geetools\ipymap.py in ()
18 get_image_tile, get_geojson_tile, get_bounds, get_zoom,
19 create_html, feature_properties_output
---> 20 import maptool
21 from . import ipytools
22 import threading

ModuleNotFoundError: No module named 'maptool'

Improve masked values of `tools.imagecollection.fill_with_last` function

keep the mask data until the end of the process and check if the resulting value is 0 and it was a masked value, mask it again, so not to keep data with 0 value

Issue running cloud masking on Sentinel Collection

Hi!
Whenever I try to run the ESA cloud masking on a sentinel collection I get the error: "object of type 'Image' has no len()"

my code is as follows:

`imagecol = ee.ImageCollection("COPERNICUS/S2_SR").filterDate('2019-02-21', '2019-03-23').filterBounds(riv);

def s2cloud(imagecol):
return imagecol.updateMask(cloud_mask.sentinel2(imagecol))

imagecol2 = imagecol.map(s2cloud);
`

I really appreciate any help- thanks for this awesome toolbox!

bugs: Map = Map()

Hi Rodrigo,
When i try your code on google colab,an error occured.
IndexError Traceback (most recent call last)
in ()
3 i = ee.Image('COPERNICUS/S2/20181122T142749_20181122T143353_T18GYT').clip(aoi)
4 grid = tools.image.toGrid(i, 3, geometry=aoi)
----> 5 Map = Map()
6
7 Map.show()

1 frames
/usr/local/lib/python3.6/dist-packages/ipygee/assets.py in init(self, map, **kwargs)
20 # Thumb height
21 self.thumb_height = kwargs.get('thumb_height', 300)
---> 22 self.root_path = ee.data.getAssetRoots()[0]['id']
23
24 # Map

IndexError: list index out of range

Could you please help me about it?

Invalid GeoJSON when using batch to export to Asset

I am trying to export a collection of Landsat 8 images as assets but keep getting a geometry error. I originally was performing a number of operations to the scenes once selected from the landsat collection and thought maybe the geometry was getting altered in the process. In the code below, I tried to simplify the process so that the final image collection represents the least cloudy Landsat 8 scene over a selection of watersheds in Montana, during the 2018growing season, but the error still occurs.

Thanks in advance for the help and I appreciate the tools you have put together!

var huc8 = ee.FeatureCollection("USGS/WBD/2017/HUC08")

var mt = ee.FeatureCollection("TIGER/2018/States").filter(ee.Filter.equals('NAME', 'Montana'));
// Map.addLayer(mt)

var sheds = huc8.filterBounds(mt).filter(ee.Filter.or(ee.Filter.equals('name','Peoples'), ee.Filter.eq('name','Belt'), ee.Filter.eq('name','Arrow')));

// var sheds = ee.FeatureCollection('users/ryanpatrickrock/bpHuc_testData')

print(sheds);

var testFun = function(feature){
  
  var ROI = feature.geometry();
  
  var addCloud = function(image) {
    var cloud = ee.Algorithms.Landsat.simpleCloudScore(image).select('cloud');
    var cloudiness = cloud.reduceRegion({ // calculate average cloud score across the ROI i.e the watershed specified
      reducer: 'mean', 
      geometry: ROI, 
      scale: 30,
      maxPixels: 1e12
    });
  return image.set(cloudiness); // add this cloud score to the metadata of the image
  };
  
  var l8 = ee.ImageCollection("LANDSAT/LC08/C01/T1_TOA")
        .filterDate('2018-05-15', '2018-09-15')
        .filterBounds(ROI)
        .map(addCloud)
        .sort('cloud')

  var scene = l8.first()
                .select(['B1','B2','B3','B4','B5','B6','B7'])
                .clip(ROI)
  return scene
};

var sceneCol = ee.ImageCollection(sheds.map(testFun));
print(sceneCol)

Map.addLayer(sceneCol, {bands: ['B4', 'B3', 'B2'], min: 0, max: 0.5})

Map.addLayer(sceneCol.first().geometry().bounds())

print(sceneCol.first().geometry().bounds())

var batch = require('users/fitoprincipe/geetools:batch');

batch.Download.ImageCollection.toAsset(sceneCol, 'users/ryanpatrickrock', {
  name: 'test_batchCollection',
  scale: 30,
  region: null,
  maxPixels: 1e12
  }
);

Table Upload in the assets

Hi folks,

I am trying to upload a table in my asset, however, I couldn´t do it. I selected the dbf, shp, and shx files as described the GEE. after minutes of running the process, I received the warning message: Error: Unable to parse the specified file.

Can anyone help me?

batch.imagecollection.toCloudStorage does not upload all files

Thank you for making this functionality available.

I'm been trying to export some images on my google cloud storage, but am running into trouble.
Below is an example of my run when I try to upload 9 images

BUCKET = "detect_proto"
imageExportFormatOptions = {
      'patchDimensions': [75, 75],  
      'maxFileSize': 100000000,    
      'compressed': True
    }
batch.imagecollection.toCloudStorage(newImgCol, bucket=BUCKET, folder="geetools", scale=10, 
                                     fileFormat = 'TFRecord', verbose=True,
                                     formatOptions = imageExportFormatOptions)

This will yield the following output

adding task 0 to list
adding task 1 to list
adding task 2 to list
adding task 3 to list
adding task 4 to list
adding task 5 to list
adding task 6 to list
adding task 7 to list
adding task 8 to list

But when I check my cloud storage, only 1 file has been uploaded

gsutil ls gs://detect_proto/geetools

gs://detect_proto/geetools/0.json
gs://detect_proto/geetools/0.tfrecord.gz

I'm a bit confused why only 1 file has been uploaded, any thoughts on how to investigate this would be greatly appreciated.
CC @MarcCoru @fitoprincipe

Issues when using geetools to download imagecollection

https://gis.stackexchange.com/questions/279412/exporting-large-number-of-landsat-images-from-google-earth-engine
I followed your answer to this issue(downloading imageCollection from GEE), but I got another problem. Here is my code:

download the data

ndvi = ee.ImageCollection('MODIS/006/MOD13A2').filterDate('2009-01-01','2017-12-31').select('NDVI')

ndvi_mon = ndvi.filter(ee.Filter.calendarRange(2009,2017,'year')).filter(ee.Filter.calendarRange(6,7,'month'))

from geetools import tools

download image

tasks = tools.col2drive(ndvi_mon, 'NDVI0607', scale=1000)

try n° 0 ERROR: name 'TYPES' is not defined
try n° 1 ERROR: name 'TYPES' is not defined
try n° 2 ERROR: name 'TYPES' is not defined
try n° 3 ERROR: name 'TYPES' is not defined
try n° 4 ERROR: name 'TYPES' is not defined

Even though I added (dataType='float'), it is still the same error.

Cannot find col2drive in geetools.tools

Hi Rodrigo,

I am also trying to use geetools to download imageCollection. However, I cannot find col2drive function in geetools.tools. My code is below. Thank you for your help and this powerful tool.

from geetools import tools import ee ee.Initialize() S1_images=ee.ImageCollection('COPERNICUS/S1_GRD').filterDate('2017-08-27','2017-08-29') task = tools.col2drive(S1_images,r'D:\result')
I got the error message
AttributeError: module 'geetools.tools' has no attribute 'col2drive'

Move layers up and down in ui.Map not working in py3

Works fine in python 2

addlayer for feature won't work using ipymap?

Hi

great tools , thanks a lot! Is it possible that addLayer cannot show Features on map, only geometries? I got warning message TypeError: addGeoJson() takes exactly 2 arguments (1 given) on a Feature. When using on FeatureCollection, I get: addLayer doesn't support adding the specified object to the map.

See:

import ee
ee.Initialize()
rectangle = ee.Geometry.Rectangle([-40, -20, 40, 20]);
rectangle_f = ee.Feature(rectangle)
rectangle_fc = ee.FeatureCollection(rectangle_f)

from geetools import ipymap
Map2 = ipymap.Map()

Map2.show()
Map2.addLayer(rectangle, {}, 'rectangle')
Map2.centerObject(rectangle)

All works fine. But now these two will throw an error?

Map2.addLayer(rectangle_f, {}, 'rectangle feature')
Map2.addLayer(rectangle_fc, {}, 'rectangle feature')

Duplicated folder when downloading an ImageCollection to Google Drive

Hello @fitoprincipe , Would you be so kind as to help me?

I am interested in downloading Sentinel-1 images using this script:

import ee
ee.Authenticate()
ee.Initialize()

import geetools

region = ee.Geometry.Rectangle([-64.1569659949999931,-8.8276458170000005, 
                                -63.7711738149999974,-8.5146591039999997])

start_date = ee.Date('2019-05-01')
finish_date = ee.Date('2019-05-7')
orbit = 'ASCENDING'

collectionS1 = ee.ImageCollection('COPERNICUS/S1_GRD')\
    .filter(ee.Filter.listContains('transmitterReceiverPolarisation', 'VV'))\
    .filter(ee.Filter.eq('instrumentMode', 'IW'))\
    .filterDate(start_date, finish_date)\
    .filterBounds(region)

collectionVV = collectionS1.select('VV')

folder = 'GEE'

geetools.batch.imagecollection.toDrive(collectionVV, folder=folder, scale=10, maxPixels=1e13,
                 region=region)

It works well, however two folders with the same name are downloaded, and the total of the downloaded images are in both folders.

Why does this happen?

Thank you so much for your time

Best,
Yonatan

Originally posted by @ytarazona in #44 (comment)

region parameter in `batch.ImageCollection.toAsset`

This parameter has to be modified to match the new structure

bug: Missing return in batch.Export.imagecollection.toDrive

Documentation saids that the function will return a list of tasks

:return: list of tasks
:rtype: list

but there is not return at the end of the function.

As the list of tasks is already created, it is just necessary to add :

return tasklist

at the end of the function

Batch download stopped working

Hi,
I've been using this code for a long time, and today it stopped working https://code.earthengine.google.com/2e38ff1b7dc0d006715a085cb469ec90 . If shortly:

1. I upload the Sentinel-2 dataset,
2. do cloud masking and calculate vegetation indices,
3. calculate the number of valid pixels within the region and filter out images with less than 100 valid pixels.
4. batch download of the resulted vegetation indices.

Today, batch download stopped working with step 3: I get notification "An internal error has occurred (request: 44215952-a82b-4627-9be7-844555231e66)"; at the same time, the batch download runs only for 2-5 files (out of 12).
If I skip step 3, the code runs properly and the batch download works fine. I have reported this problem in the GEE feedback field, but I don't know how long it will take. Maybe you could help me understand the reason for the problem and provide some solution?
I really appreciate any help you can provide.

pip install geetools ERROR: ImportError: No module named shapefile

I am trying to install the Geetools library on datalab in docker, and the error occurs:
ImportError: No module named shapefile.
Someone has had something similar.
thank you!!

Not equal date interval are shown as equally distanced in Chart

When plotting a time serie the x axis is not properly distributed. All dates are plotted equally distanced while they are not.

It's clear in the first 3 dates, there are 7 days between the first and the second, and just minutes between the second and the third, but are plotted equally distanced

Using maxImgs parameter gives EEException

If I include the maxImgs parameter in a call to batch.Export.imagecollection.toDrive, I get the following error: EEException: Unknown configuration options: {'maxImgs': 20}..

Cloud Mask

Hi Rodrigo!

It seems very interesting the library you've written. I'm looking for more information on how to use the cloud mask functionality. Could you give me an example?

Different results from reduceRegion after running mosaicSameDay utility

I am not sure if this is a bug, but I'll report it and you can hopefully determine. I am using the python API and gee_tools. I have shared my shapefile asset so hopefully you can just plug this in.

I am computing the area of a Landsat image within a polygon, and then computing it again after using the mosaicSameDay tool. The image I'm using completely covers the polygon, so one would expect the area to be the same before or after running the tool. However, this is not the case and the areas are substantially different.

import ee
from geetools.tools import imagecollection as ictools

# function to compute overlap area between polygon and image
def valid_area(img):
    """
    Returns a feature containing the total valid area (as defined by the 
    append_vo_band function) within an image. Requires a 'valid_obs' band to
    be present in an image.
    """
    
    # Convert image to binary
    bin_im = ee.Image(img.select('B1'))
    bin_im = bin_im.where(bin_im.neq(0), 1)
    bin_im = bin_im.where(bin_im.neq(1), 0)
    # Convert valid observation band to pixel areas band (no valid obs are still zeros)
    pix_areas = ee.Image.multiply(bin_im, ee.Image.pixelArea())
    
    # Number of pixels in valid region
    validarea = pix_areas.reduceRegion(
            reducer = ee.Reducer.sum(),
            geometry = feat.geometry(),
            scale = scale,
            maxPixels = 1e12,
            bestEffort = False)
        
    return ee.Feature(None, {'valid_area':validarea.get('B1')})

# Set things up
ee.Initialize()
start_date = '2010-01-01'
end_date = '2019-01-01'
scale = 30 # The scale to do the analysis
n_images = 10 # Maximum number of images to fetch--used for testing
provinces = ee.FeatureCollection('users/jonschwenk/Ecuador_provinces')
feat = provinces.first() # work on first polygon

# Landsat 8 analysis
L8 = ee.ImageCollection('LANDSAT/LC08/C01/T1_TOA') \
    .filterDate(start_date, end_date) \
    .filterBounds(feat.geometry()) \

L8_m = ictools.mosaicSameDay(L8)

# Now check results
L8_m = ee.ImageCollection(L8_m.toList(n_images))

L8area = valid_area(L8.first()).getInfo()['properties']['valid_area']
L8marea = valid_area(L8_m.first()).getInfo()['properties']['valid_area']

print(L8area) shows 3,943,647,359,
while print(L8marea) shows 3,161,128.

To reiterate: the L8 image completely covers the polygon (and the area of the polygon is roughtly 3.9x10^9, matching the L8area value), so mosaicing additional images to it should not change its area value.

I also know that the images come from the same date, so my question is: why is the area different after running the sameDayMosaic tool? My best guess is that the tool does not preserve the CRS or CRS transform, although I thought GEE would handle those things under the hood through the ee.Image.pixelArea() function.

Let me know if there's any more info you need :)

Have you seen this weird error before?

`---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
in
1 from geetools import ui
2
----> 3 Map = ui.Map()
4 Map.show()

~/anaconda3/lib/python3.7/site-packages/geetools/ui/ipymap.py in init(self, tabs, **kwargs)
79
80 # Asset Manager Widget
---> 81 asset_manager = ipytools.AssetManager(self)
82
83 # Layers

~/anaconda3/lib/python3.7/site-packages/geetools/ui/ipytools.py in init(self, map, **kwargs)
310 # Thumb height
311 self.thumb_height = kwargs.get('thumb_height', 300)
--> 312 self.root_path = ee.data.getAssetRoots()[0]['id']
313
314 # Map

IndexError: list index out of range`

ERROR showing info of Feature

When clicking with the inspector tool over a Feature, shows:

'ascii' codec can't encode character u'\ufffd' in position 1: ordinal not in range(128)

Exporting a Collection into Local Computer Disk

HI!,

I would like to download an Image collection (more than 80 GB) into my personal computer, because google drive doesn't offer a lot of storage.

Is that possible? and if yes, How?

Thank you

Check the number of task in the queue

I am a user of gee_tools. I usually export image collection to google drive. However, when the number of the task in the queue is large than 3000, the program will be halted. I wonder if there is any way to check the number of the task in the queue and pause the program when the number is large?

Thanks for your kind help and looking forward to your reply.

Best,
Enze

B10 in sentinel is not available

I tried running hollsteinMask but it requires B10 in sentinel2. It seems this was removed from the GEE collection.

An issue with hollstein_S2

Dear Rodrigo,

Thank you very much for your gee_tools library for python. I am trying to utilize it into my study.

It is okay to map with s2ESA (cloud_mask.sentinel2) without error, however when I map with s2holl (cloud_mask.hollstein_S2), my app produce this error: Invalid argument specified for ee.List(): hollstein

Could you please guide me how to fix? Or did I do something wrong?

Thank you!

Invalid GeoJSON geometry error

Hello!
I received a similar error message as reported on fitoprincipe/geetools-code-editor#2 when trying to batch download an image collection, except on python.
Here's the code:

`import ee
ee.Initialize()
from geetools import batch

feature = ee.Geometry.Polygon([[-20.3014885, -51.5035369], [-20.84233950, -51.6313196], [-20.6922002, -51.2336120], [-20.5804476, -52.1965132]]), {'site': 1}
fc = ee.FeatureCollection(feature)
collection = ee.ImageCollection("COPERNICUS/S5P/NRTI/L3_CLOUD").select('cloud_optical_depth').filterDate('2019-09-10', '2019-09-13').filterBounds(fc.geometry())
image = collection.mosaic()

tasklist = batch.image.toDriveByFeature(image, collection=fc, property='site', folder='Result_test_S5P', name='test', scale=305, dataType='float')

tasklist.start()`

And the Error:
`---------------------------------------------------------------------------
EEException Traceback (most recent call last)
in
6
7 feature = ee.Geometry.Polygon([[-20.3014885, -51.5035369], [-20.84233950, -51.6313196], [-20.6922002, -51.2336120], [-20.5804476, -52.1965132]]), {'site': 1}
----> 8 fc = ee.FeatureCollection(feature)
9 collection = ee.ImageCollection("COPERNICUS/S5P/NRTI/L3_CLOUD").select('cloud_optical_depth').filterDate('2019-09-10', '2019-09-13').filterBounds(fc.geometry())
10 image = collection.mosaic()

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\computedobject.py in call(cls, *args, **kwargs)
30 return args[0]
31 else:
---> 32 return type.call(cls, *args, **kwargs)
33
34

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\featurecollection.py in init(self, args, opt_column)
62 super(FeatureCollection, self).init(
63 apifunction.ApiFunction.lookup('Collection'), {
---> 64 'features': [feature.Feature(i) for i in args]
65 })
66 elif isinstance(args, ee_list.List):

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\featurecollection.py in (.0)
62 super(FeatureCollection, self).init(
63 apifunction.ApiFunction.lookup('Collection'), {
---> 64 'features': [feature.Feature(i) for i in args]
65 })
66 elif isinstance(args, ee_list.List):

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\computedobject.py in call(cls, *args, **kwargs)
30 return args[0]
31 else:
---> 32 return type.call(cls, *args, **kwargs)
33
34

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\feature.py in init(self, geom, opt_properties)
75 # turning out to be GeoJSON.
76 super(Feature, self).init(feature_constructor, {
---> 77 'geometry': geometry.Geometry(geom),
78 'metadata': opt_properties or None
79 })

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\computedobject.py in call(cls, *args, **kwargs)
30 return args[0]
31 else:
---> 32 return type.call(cls, *args, **kwargs)
33
34

~\AppData\Local\Continuum\miniconda3\envs\S5P-37\lib\site-packages\ee\geometry.py in init(self, geo_json, opt_proj, opt_geodesic, opt_evenOdd)
81
82 if not Geometry._isValidGeometry(geo_json):
---> 83 raise ee_exception.EEException('Invalid GeoJSON geometry.')
84
85 super(Geometry, self).init(None, None)

EEException: Invalid GeoJSON geometry.`

I also tried importing the geometry as shapefile and as a geojson through geopandas, still got the same error. Not sure if this is a gee-tools error or an ee error. Can you help me?

I am new to python so hopefully it's just me doing something wrong...
Thanks in advance.

band name error

cloud masking leads into this error

"
HttpError: <HttpError 400 when requesting https://earthengine.googleapis.com/v1alpha/projects/earthengine-legacy/value:compute?prettyPrint=false&alt=json returned "Image.select: Pattern 'pixel_qa' did not match any bands.". Details: "Image.select: Pattern 'pixel_qa' did not match any bands.">"

is it due to Landsat SR band name convention changed as 'QA_PIXEL',

Export to Drive

Hello,

I've tried exporting a couple of different ways but seem to have no luck. I tried using the exportByFeat.ipynb as a guide and followed the script as is but I get the same result. The correct folder is created in my drive but there is nothing inside. Am I doing anything wrong?

new to GEE python API

Hi there -
Sorry to bug you with, what I am guessing, is a very basic question! I've been trying to piece together your code snippets from various StackExchange posts, but am likely missing something obvious. I'm using Spyder and have imported your gee_tools.
My task is simple! I want to download multiple L8 images (as many as available) of a small region in Greenland but am having trouble with the iterating part. I think I "should" be able to call your gee.ExportCol command as well, but I'm not sure where that fits in.

kgl = ee.Geometry.Polygon(
[[[-33.18145751953125, 68.63655326841952],
[-32.79144287109375, 68.52722953172731],
[-32.65960693359375, 68.58245935169703],
[-32.95623779296875, 68.68951879149195]]])
start = '2016-05-01'
stop = '2016-06-01'
images = ee.ImageCollection("LANDSAT/LC8_L1T_TOA")
.filterBounds(kgl)
images = images.filterDate(start, stop)
listOfImages = images.toList(images.size());

After this snippet, I tried lots of variations of the task= ee.batch.Export.image.toDrive and gee.ExportCol...but I don't think I have my list set up correctly. Would appreciate any suggestions!

batch = require('users/fitoprincipe/geetools:batch')

Hi,

I'm contacting you because I have a problem with geetools.
I'm working on jupyter with anaconda 3.

Here my code :

## Init ##
from IPython.display import Image
import ee
import folium
from geetools import batch
import datetime

ee.Initialize()

parcelle = ee.Geometry.Rectangle([[xxxxxxx,xxxxxx],[xxxxxxx,xxxxxxx]])

def addNDVI(image):
    ndvi=(image.expression('float(b("B4") - b("B3")) / (b("B4") + b("B3"))')
    .rename('NDVI'))
    return image.addBands(ndvi)

collection = (ee.ImageCollection('LANDSAT/LE07/C01/T1_SR')
              .filterDate(datetime.datetime(2000,1,1),
                         datetime.datetime(2020,3,3))
              .filterBounds(parcelle)
              .select(['B4', 'B3'])
              .map(addNDVI).select('NDVI'));

batch.ImageCollection.toDrive(collection, 'Landsat7',
                                      {scale: 30,
                                      region: parcelle.getInfo()["coordinates"],
                                      type: 'float'})

And I'm getting the following error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-7-fde281556d5e> in <module>
----> 1 batch.ImageCollection.toDrive(collection, 'Landsat7',
      2                                       {scale: 30,
      3                                       region: parcelle.getInfo()["coordinates"],
      4                                       type: 'float'})

AttributeError: module 'geetools.batch' has no attribute 'ImageCollection'

Could you help me ?

How can i upload assets in Jupyter?

Are there corresponding functions in gee_tools?

Empty images (NaN values pixel) and incomprehensible file names

Hello!
I have some dataset. I download pictures from this dataset.
I use gee-tools to export collections of images.
However, some downloaded files have values of all pixels NaN or have names of this kind and also have values of all pixels NaN
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000000000-0000000000.tif
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000000000-0000006912.tif
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000006912-0000000000.tif
Altaiskii_krai__Nikolaevka_2018-10-15_S2TOA-0000006912-0000006912.tif

I learned to recognize such "zero" pictures by their volume. But I want to understand the reason for this. I want to remove such pictures from export.

Here's my code.

sat = geedatasets.sentinel.Sentinel2SR()
s2toa = geedatasets.fromShortName('S2TOA')

##collection = s2toa.collection().filterBounds(point)
#collection = collection.filterDate(start, end)
#collection = collection.filterMetadata("CLOUD_COVERAGE_ASSESSMENT", 'less_than', 20.0)

visParam = {'bands':['B4', 'B3', 'B2'], 'min':0, 'max':5000}
vis = sat.visualization('TrueColor')#('NSR')
counImages = 0

def get_dates(col):
dates = ee.List(col.toList(col.size()).map(lambda img: ee.Image(img).date().format()))
return dates

TotalSizeImageCollection = ee.Number(0)
sel = 2
for q in arrNump:
if(validate(q[3]) == True):
if (isinstance(q[6], str)):
q[6] = transliterate(q[6])
if (isinstance(q[7], str)):
q[7] = transliterate(q[7])

listPoint = ee.List([float(q[6])]).add(float(q[7]))

#listPoint = ee.List([55.958727]).add(54.735147)
p = ee.Geometry.Point(listPoint)
site = (p.buffer(30000)).bounds()

def clp(img):
  return img.clip(site)

startDate = ee.Date(formatterdate(q[3])).advance(-10,'day')
if(isinstance(q[2], int)):
  endDate = ee.Date(startDate.advance(10,'day'))
else:
  if(validate(q[2]) == True):
    endDate = ee.Date(formatterdate(q[2])).advance(10,'day')

col = sat.collection().filterBounds(site).filterDate(startDate, endDate).filterMetadata("CLOUD_COVERAGE_ASSESSMENT", 'less_than', 30.0)   

countMos = 0

if(col.size().getInfo()>0):
  countMos = col.size().getInfo()

global counImages
counImages = counImages + countMos
for i in range(col.size().getInfo()):
  mosaics = tools.imagecollection.mosaicSameDay(col)
  mosaics = mosaics.map(clp)

region = transliterate(q[0])
city = transliterate(q[1])

scale = 10
name_pattern = region + '_'+ city + '_' + '{system_date}_{sat}' #{system_date}_{sat}_{WRS_PATH:%d}-{WRS_ROW:%d}
date_pattern = 'Y-MM-dd' # dd: day, MMM: month (JAN), y: year
folder = region
data_type = 'float'
extra = dict(sat='S2TOA')
region = site

print(sel, q[3], q[2], q[1], q[0])

if(sel>110): 
  tasks = geetools.batch.Export.imagecollection.toDrive(collection=mosaics, folder=folder, region=region, namePattern=name_pattern, scale=scale, dataType=data_type, datePattern=date_pattern, extra=extra, verbose=True, maxPixels=int(1e13))

sel = sel + 1

if(sel>120):
break
print(sel)

pip install geetools ERROR

When I tryng to install the package i get out this issue:

Collecting geetools
Using cached https://files.pythonhosted.org/packages/1d/57/f4d913a480480d44cfa94c78911b935dd40df2a2135be85d529741b21b91/geetools-0.1.1.tar.gz
Command "python setup.py egg_info" failed with error code 1 in C:\Users\xxxxx\AppData\Local\Temp\pip-install-ajxqlt3t\geetools
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 1, in
File "C:\Users\xxxxxx\AppData\Local\Temp\pip-install-ajxqlt3t\geetools\setup.py", line 5, in
from geetools import version
File "C:\Users\xxxxxxx\Local\Temp\pip-install-ajxqlt3t\geetools\geetools_init_.py", line 20, in
from .ui.ipymap import Map
File "C:\Users\GE899~1.DEL\AppData\Local\Temp\pip-install-ajxqlt3t\geetools\geetools\ui_init_.py", line 12
def eprint(eeobject, indent=2, notebook=False, async=False):
^
SyntaxError: invalid syntax

MODIS projection?

Hi Rodrigo -

Thanks again for your earlier help. Have you done any work with MODIS?

I realize that using python to batch download images means that MODIS gets downloaded in its original (annoying) sinusoidal projection. My attempts with gdalwarp and gdal_translate after downloading the images give me a syntax error:

vel='/Users/leigh/Google Drive/GEE/KGL/MODQ/MOD09GQ_005_2017_03_26.tif'
gdalwarp -s_srs '+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs' vel vel_out.tif

Is there a way to reproject it before downloading? Thanks, and sorry to bug you again!

• leigh

Preserve band datatype in mosaicSameDay utility

The mosaicSameDay function in imagecollection.py is pretty handy, but it fails to maintain the proper datatypes in the output mosaic image bands. Instead, each band becomes a Float (at least when I tried with Sentinel 2). Would it be possible to map the band datatype from the original image to the mosaiced one?

mosiacSameday

Ubuntu 18.04 installation fails

# pip install geetools
Collecting geetools
  Using cached https://files.pythonhosted.org/packages/51/83/3f6f6cc185902d27468dcbef69736d12244386ec26439cabb6a09a30f471/geetools-0.1.7.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-bitz479n/geetools/setup.py", line 5, in <module>
        from geetools import __version__
      File "/tmp/pip-build-bitz479n/geetools/geetools/__init__.py", line 17, in <module>
        from . import tools, bitreader, cloud_mask, expressions, decision_tree,\
      File "/tmp/pip-build-bitz479n/geetools/geetools/tools/__init__.py", line 3, in <module>
        from . import imagecollection, date, dictionary, image, number, \
      File "/tmp/pip-build-bitz479n/geetools/geetools/tools/imagecollection.py", line 3, in <module>
        import ee
      File "/usr/local/lib/python3.6/dist-packages/ee/__init__.py", line 1, in <module>
        from .main import main
      File "/usr/local/lib/python3.6/dist-packages/ee/main.py", line 10, in <module>
        import StringIO
    ModuleNotFoundError: No module named 'StringIO'
    
    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-bitz479n/geetools/

likewise with pip2, or pip3

AttributeError: type object 'Export' has no attribute 'ImageCollection'

Hi! You are saving my life with these tools. Thank you in advance!
Sorry to ask but, I keep getting this error at the moment of exporting 10 images to my Drive. My code looks like this:

from geetools import batch
import ee
ee.Initialize()
zona_aire = ee.FeatureCollection("users/romeldrivera/zona_aire")
start = ee.Date.fromYMD(2018,1,1)
days = ee.List.sequence(0,9)
def one(d):
    return start.advance(d,'day')
startDates = days.map(one)
def three(img):
    return img.clip(zona_aire)
def two(m):
    start = ee.Date(m)
    end = ee.Date(m).advance(1,'day')
    date_range = ee.DateRange(start,end)
    MODISday = ee.ImageCollection('MODIS/006/MCD19A2_GRANULES').select('Optical_Depth_047').filterDate(date_range).map(three)
    return(MODISday.median())
daymap = two
list_of_images = startDates.map(daymap)
mt = ee.ImageCollection(list_of_images)
tasks = batch.Export.ImageCollection.toDrive(mt, 'AOD2018', scale=30, region=zona_aire.getInfo()["coordinates"])

I run that code and get

tasks = batch.Export.ImageCollection.toDrive(mt, 'AOD2018', scale=30, region=zona_aire.getInfo()["coordinates"])
AttributeError: type object 'Export' has no attribute 'ImageCollection'

I appreciate your time and attention!

batch.Download exports empty Tif

I am using batch.Download to export each image from a NDVI MODIS collection, however the output is a black .tif with only ~18KB. Here is my code:

Here is my Tif:

Do you know what could be the reason for this?
Thank you!

Error in uploading shapefile

Hello,
I am new user of GEE, I am trying to perform a supervised classification, so when I import my shapefile, I get the following error message : "Error: Unable to transform edge (549555.928576868,3474668.316669156 to 549555.9286273334,3474668.3166489694) from Planar EPSG:4326 to Geodesic EPSG:4326 (within 4.488328883882435E-5 units)"
I didn't understand, how can I transform it from Planar EPSG:4326 to Geodesic EPSG:4326 ?

Any help, Please?

AyouCha

setup

python 3.6 is not supported

python 3.6 is not supported

Import error in Python 3.4

It works fine in python 3.5, but in python 3.4 when doing from geetools import ui it throws an error:

ImportError Traceback (most recent call last)
in ()
----> 1 from geetools.ui import ipymap

~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/init.py in ()
4 import threading
5 import pprint
----> 6 from . import chart, imagestrip, ipymap, ipytools, maptool, dispatcher
7
8 from IPython.display import display

~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/ipymap.py in ()
14 from .maptool import inverseCoordinates, getImageTile, getGeojsonTile,
15 getBounds, getZoom, featurePropertiesOutput
---> 16 from . import maptool, ipytools
17 import threading
18 from copy import copy

~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/ipytools.py in ()
7
8 from .. import batch
----> 9 from .dispatcher import widgetDispatcher
10
11 # imports for async widgets

~/Projects/eds_vietnam/env/eds_vietnam/lib/python3.4/site-packages/geetools/ui/dispatcher.py in ()
3 import ee
4 from ipywidgets import *
----> 5 from . import ipytools
6
7 def belongToEE(eeobject):

ImportError: cannot import name 'ipytools'

ImageCollection Export to Cloud Storage

The ImageCollection Export to Drive function seems useful. I think it would be useful to add similar methods for other export destinations. My particular use case requires ImageCollection Export to Cloud Storage, which doesn't seem to be implemented here or on base EE. There may be other export methods missing from EE that would be useful here as well.

(Also, thanks for the neat library!)