planetlabs/planet-client-python

Python client for Planet APIs

Planet SDK for Python

The Planet Software Development Kit (SDK) for Python provides both a Python API and a command-line interface (CLI) to make use of the Planet APIs. Everything you need to get started is found in our online documentation.

Version 2.0 includes support for the core workflows of the following APIs:

• Data - Search for imagery from Planet's data catalog.
• Orders - Process and download or deliver imagery.
• Subscriptions - Set up a search to auto-process and deliver imagery.

After the initial 2.0 release there will be additional work to support the remaining Planet APIs: basemaps, tasking) and analytics.

Versions and Stability

The default branch (main) of this repo is for the Planet SDK for Python, a complete rewrite and upgrade from the original Planet Python Client. If you are looking for the source code to that library see the v1 branch.

The Planet SDK for Python is in 'pre-release' stages, working towards a solid beta release in December. Upcoming milestones are tracked in the Planet SDK for Python Milestones.

Installation and Quick Start

The main installation path and first steps are found in the Quick Start Guide of the documentation.

Installing from source

This option enables you to get all the latest changes, but things might also be a bit less stable. To install you must clone the planet-client-python repository to your local computer. After you have the repo local just navigate to the root directory, where this readme lives.

Then you can install locally with pip:

$ pip install . 

Documentation

Documentation is currently hosted online It should be considered 'in progress', with many updates to come. It can also be built and hosted locally (see CONTRIBUTING.md) or can be read from source in the docs directory.

Authentication

Planet's APIs require an account for use. To get started you need to Get a Planet Account.

Development

To contribute or develop with this library, see CONTRIBUTING.md.

planet-client-python's Issues

sync uses acquired field, should use new published field

Example on how to use client to download images/tiles for AOI

Hello,

I'm not able to find any code exemplifying how to use Planet's API client to download images. I have been able to pull information on assets available given a set of filters on AOI, dates etc. using the (only) example in the ./examples folder on this repo. But is there any documentation on how to download an item of interest via the Planet API client?

Thanks,
Adrian

Use futures only

Per @ischneider suggestion, the library uses requests-futures for a subset of requests, let's send all requests via requests-futures.

Allow mosaic search based on AOI

The mosaic-quads command should support AOI intersection like planet search does since the API also supports this: https://www.planet.com/docs/v0/mosaics/#get-mosaic-quads.

Open source

Let's get this moving sooner rather than later. One prerequisite might be to reach parity with the JavaScript client that Tim has been working on.

/cc @ischneider @tschaub @warmerdam

handle API error messages better on client

by default, format API responses for better readability. allow a flag to get the raw error as JSON.

Example:

{"message": "Query parameter validation failed", "errors": [{"field": "intersects", "message": "GeometryCollections are currently unsupported"}]}

In this case, the provided AOI is invalid. Ideally, the error 'translator' would see that intersects field is bad and (since we built the query from the provided API) note the provided AOI is invalid with the reason. If we cannot determine a more specific message, we can default to formatting like so:

Query parameter validation failed:
Field 'intersects' invalid:  GeometryCollections are currently unsupported

Also see #24 (re: raw error flag)

add intersects support to mosaic queries once supported in API

currently not supported

better Windows experience

• check if recent pex versions improve compatibility (errors running w/ current pex 'binary')
• pywin32 dependency

erroneous 'None' appearing in CLI error for missing argument

for example planet sync yields

Usage: planet sync [OPTIONS] DESTINATION

Error: Missing argument "destination".None

see pallets/click#312

Add mosaic resources

ctrl+c to interupt download of scenes gives you aborted! message but won't give you back control of the console

alternate means to find api key

Looking for a dot-file or other local config[1] that specifies the key. Thoughts?

For now, a nice way to auto-enable is to add to virtualenv setup

The idea of this hook was to add fall-backs:

planet-client-python/planet/api/__init__.py

Line 68 in ae3e4ba

def _find_api_key():

related: #5

[1] http://standards.freedesktop.org/basedir-spec/basedir-spec-latest.html

consider scheduling for sync command

perhaps using https://github.com/dbader/schedule or a similar approach with fewer options

an existing alternative is using cron

paged response broken (was: example broken)

python examples/download-tiffs.py

Traceback (most recent call last):
  File "examples/download-tiffs.py", line 55, in <module>
    for s in scene.iter(pages=3):
  File "planet/api/models.py", line 138, in iter
    page = page.next()
  File "planet/api/models.py", line 128, in next
    request = Request(next, body_type=Scenes)
TypeError: __init__() takes at least 3 arguments (3 given)

refactor/extract sync command

Does too much
CLI should be a thin wrapper to the guts
Should be more testable

need to adjust login status checking

400: invalid parameters
401: invalid credentials

download scene has stopped working

Appears the client is not passing authorization headers when following a redirect.

more options in sync command

thumbnails, analytic, etc.

once query by workspace is supported, use it

This means instead of building query params from the workspace, we'll just use the workspace id as a query parameter.

signal handling broken on 2.7 with requests_futures

Ctrl-C has no effect and async tasks proceed. See: agronholm/pythonfutures#25

Unless we can work-around with requests_futures or the futures backport is fixed , might need to use a multiprocessing approach.

planet metadata multiple sids

would be nice to be able to pass multiple scene ids into planet metadata, returning a featurecollection just like planet search.

Raise more appropriate exception when api key not defined

When the api key is not defined, client._get proceeds to construct the authorization header. Either

1. api key existence should be enforced
2. an authorization error from the API should propagate

Either of the above is more informative than the TypeError.

pplanet@ip-172-31-27-162:~/planet_common$ planet get-scenes-list
Traceback (most recent call last):
  File "/home/planet/anaconda/bin/planet", line 9, in <module>
    load_entry_point('plapi==0.0.1.dev0', 'console_scripts', 'planet')()
  File "/home/planet/anaconda/lib/python2.7/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/home/planet/anaconda/lib/python2.7/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/home/planet/anaconda/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/planet/anaconda/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/planet/anaconda/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/home/planet/planet_common/planet-client-python/planet/scripts/__init__.py", line 103, in get_scenes_list
    res = call_and_wrap(client.get_scenes_list, scene_type=scene_type, intersects=aoi)
  File "/home/planet/planet_common/planet-client-python/planet/scripts/__init__.py", line 18, in call_and_wrap
    return func(*args, **kw)
  File "/home/planet/planet_common/planet-client-python/planet/api/__init__.py", line 132, in get_scenes_list
    return self._get('scenes/%s' % scene_type, params=params).content
  File "/home/planet/planet_common/planet-client-python/planet/api/__init__.py", line 118, in _get
    headers = {'Authorization': 'api-key ' + self.api_key}
TypeError: cannot concatenate 'str' and 'NoneType' objects

Attribute error when downloading from the CLI

Looking into this right now.

$ planet download 20150517_181918_0908 --product analytic
Traceback (most recent call last):
  File "/Users/amitkapadia/anaconda/envs/pl/bin/planet", line 9, in <module>
    load_entry_point('planet==0.0.1', 'console_scripts', 'planet')()
  File "/Users/amitkapadia/anaconda/envs/pl/lib/python2.7/site-packages/click/core.py", line 664, in __call__
    return self.main(*args, **kwargs)
  File "/Users/amitkapadia/anaconda/envs/pl/lib/python2.7/site-packages/click/core.py", line 644, in main
    rv = self.invoke(ctx)
  File "/Users/amitkapadia/anaconda/envs/pl/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/amitkapadia/anaconda/envs/pl/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/amitkapadia/anaconda/envs/pl/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/Users/amitkapadia/pl/planet-client-python/planet/scripts/__init__.py", line 228, in fetch_scene_geotiff
    api.write_to_file(dest))
AttributeError: 'module' object has no attribute 'write_to_file'

Client.get_mosaic_quads(intersects = '{smallish-valid-geojson}') causes 502

I'm getting consistent bad gateway errors via the python client. when querying the get_mosaic_quads api. I think the error is coming from the get endpoint specifically.

import json
import planet.api

pl_client = planet.api.Client()

geojson_to_load = '''{"type":"FeatureCollection","features":[{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-120.925568,37.521912,0],[-120.92986,37.521572,0],[-120.949172,37.521844,0],[-120.949515,37.503326,0],[-120.925397,37.503326,0],[-120.925568,37.521912,0]]]},"properties":{}}]            }'''
parsed_geojson = json.loads(geojson_to_load)
pl_client.get_mosaic_quads(name = "re_california_summer_mosaic") # All is good here!
pl_client.get_mosaic_quads(name = "re_california_summer_mosaic", intersects = parsed_geojson)

exception_printed_out = '''
APIException: 502: <html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
'''

pl_client.get_mosaic_quads(name = "re_california_summer_mosaic", intersects = geojson_to_load)

# This throws the same exception

It appears to be an issue with the get endpoint which the python client uses but not the post endpoint.

# Here's a curl (auth header removed) that exhibits the error
curl 'https://api.planet.com/v0/mosaics/re_california_summer_mosaic/quads/?intersects=%7B%22type%22%3A%22FeatureCollection%22%2C%22features%22%3A%5B%7B%22type%22%3A%22Feature%22%2C%22geometry%22%3A%7B%22type%22%3A%22Polygon%22%2C%22coordinates%22%3A%5B%5B%5B-120.925568%2C37.521912%2C0%5D%2C%5B-120.92986%2C37.521572%2C0%5D%2C%5B-120.949172%2C37.521844%2C0%5D%2C%5B-120.949515%2C37.503326%2C0%5D%2C%5B-120.925397%2C37.503326%2C0%5D%2C%5B-120.925568%2C37.521912%2C0%5D%5D%5D%7D%2C%22properties%22%3A%7B%7D%7D%5D%20%20%20%20%20%20%20%20%20%20%20%20%7D' -H 'Authorization: Basic ${CENSORED}'


<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.8.0</center>
</body>
</html>


curl 'https://api.planet.com/v0/mosaics/re_california_summer_mosaic/quads/' -H 'Authorization: Basic CENSORED' --data-binary '{"intersects":{"type":"FeatureCollection","features":[{"type":"Feature","geometry":{"type":"Polygon","coordinates":[[[-120.92986,37.521572,0],[-120.949172,37.521844,0],[-120.949515,37.503326,0],[-120.925397,37.503326,0],[-120.925568,37.521912,0]]]},"properties":{"name":"MyPolygon"}}]}}'

{"type": "FeatureCollection",... large happy planet response here}

deal with urllib3 ssl warnings

these look like:

InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail...

Does not effect functionality but is noisy.

https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning

some CLI operations limited by lack of paging support

For example, scenes list, as currently implemented, will only return a single page of results.

This raises some questions:
Is there a practical limit we should put in place to avoid accidentally 'paging the world'? For example, a search with no filters will yield +100,000 results. One approach might be to warn/abort/limit if no AOI or limit is provided.

Command name consistency

To list all mosaics, use mosaics and to get details about one, use mosaic.

To list all worksapces, use list-workspaces and to get details about one, use get-workspace.

This could be more consistent.

Get mosaic quad info by ID

The only way to get quad info is by using mosaic-quads but it doesn't allow you to select the quad by ID.

atomic file support

when downloading files, ensure files are written atomically

planet --version

Will be useful when people want to report bugs.

$ planet --version
0.0.1

More generic search options

Ability search to take more generic inputs, like bounding boxes, for example like:

geocode -o geojson 'iowa' | jq .bbox | planet search

planet download output local filenames

Would be nice if planet download would output the local filenames for the downloaded files so that the output can be pipes to other commands

threading error

While running thumbnails command with single ID

Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 808, in __bootstrap_inner

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 761, in run

  File "/Users/REDACTED/.pex/install/futures-3.0.3-py2-none-any.whl.20b9d64ae605ce774575ed604af7a88f5586d4f7/futures-3.0.3-py2-none-any.whl/concurrent/futures/thread.py", line 65, in _worker

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/Queue.py", line 179, in get

  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 383, in notify

<type 'exceptions.TypeError'>: 'NoneType' object is not callable

Not possible to get all results of a search

The --limit parameter doesn't allow the user to set the maximum number of search results to be unlimited. It might be useful to allow the user to override all limits on the number of results since this is possible with the Python client by setting the limit to None.

without aoi, get-scenes-list yields BadQuery

@kapadia

As the default is -, this results in a server-side error since the - value is passed along to the client

Looks like an else block that sets aoi to None after the isatty check fixes this but I defer 😄

planet-client-python/planet/scripts/__init__.py

Line 91 in ff1ec2c

@- or - convention?

I'd like to nail down a practice that can be used between the Python and JS client for file/std input. I've been using a few geo tools that have adopted the - convention for accepting from stdin. For example, clipping an image to a geojson mask is accomplished with Rasterio by:

# from stdin
cat region.geojson | rio mask input.tif output.tif --geojson-mask -

# from file
rio mask input.tif output.tif --geojson-mask region.geojson

There is no option for writing inline geojson, as this quickly become verbose for a CLI command.

For the client libraries, I'd like to adopt the practices used by Rasterio, Fiona, and the USGS client for accepting from stdin or file. I do not see a compelling reason to also support inline geojson/WKT.

Currently the JS client uses the @- convention, straying from other geo tools. @ischneider has taken my input on this (thanks Ian!), so the Python client supports both @- and -. This creates ambiguity in the error message returned. For example, should a geojson file not exist:

$ planet search @aoi.geojson
Error: [Errno 2] No such file or directory: 'aoi.geojson'

the correct error is returned, thanks to the @ convention. However, since it also supports inline geometries,

$ planet search aoi.geojson
Error: BadQuery: {"message": "Query parameter validation failed", "errors": [{"field": "intersects", "message": "WKT: Unsupported geometry type"}]}

results in an ambiguous error message. This can be resolved by dropping inline geometries, and always assuming a string represents a filepath. A proper file existence check will raise the correct error.

It would be nice to avoid ambiguous error messages, but more importantly, I'd like to follow in the footsteps of rio and fio, as these are my everyday tools. Good documentation and examples around this can help ensure that others follow the same workflows that we establish.

/cc @tschaub @ischneider

add test infrastructure

using click test runner with request mocking

potential for [Errno 18] Invalid cross-device link

if atomic file write (via rename) occurs across devices. For example, if /tmp directory is mounted on a different device

the implementation should write the tmp file to the same directory as the destination

Unable to get single mosaic metadata

The planet mosaic command appears to be broken in 0.0.3:

$ planet mosaic color_balance_mosaic
Error: MissingResource: <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>404 Not Found</title>
<h1>Not Found</h1>
<p>The requested URL was not found on the server.  If you entered the URL manually please check your spelling and try again.</p>

color_balance_mosaic does appear when planet mosaics is run.

handle 502 better

as reported by a client seeing occasional 502 responses

either:

• requests should be configured to retry (if not already)
• client should attempt a delayed retry

it seems reasonable to allow the user to configure the behavior (ignore or retry X times then fail) as persistent server-side API errors are likely important in their workflows (e.g. understanding why no data showed up)

download and thumbnails cli commands do not handle missing API key well

Expected:
User sees terse error message without a stack trace

Actual:
User sees a big scary stack trace containing said message

Get workspace by name.

The workspace sync (and possibly other commands) might be more convenient if they looked up your workspace by name.

Document intersects parameter in CLI

It would be nice if the get scene list parameters, specifically intersects, were exposed in the CLI. I just had a situation where I wanted to check for scenes intersecting a specific point and had to build an entire GeoJSON file to $ cat point.geojson | planet search. A mild inconvenience, but this would have been much easier:

$ planet search --some-flag intersects 'POINT (X Y)'

paging iterator can yield None

expected behavior is that iteration completes without any None

prompt for api key and offer to store/remember

from @cholmes

I first get 'Error: InvalidAPIKey: No API key provided'. Would be cool if it could prompt me for my key then - to just enter it manually and then do the request for me (probably being able to escape out if I don't have it). Would be cooler if providing it then set the key for me, at least for that session (and ideally for more than that session).

see also #6 (the key provided at the prompt could be stored in a dot-file)

add feature iterator utility for paged geojson responses

Instead of making the API user do this manually, e.g.:

scene = client.get_scenes_list()
for s in scene.iter():
    # s is a 'page' of geojson feature collection
    scenes.extend(s.get()['features'])

this might looks like:

features = client.get_scenes_list().features()
for f in features:
  # f is geojson feature dict

Behind the scenes, the paging iterator can be used.

This should be available for all GeoJSON response types with paging.

possible invalid preparation/handling of AOI in client

reported error:

{"message": "Query parameter validation failed", "errors": [{"field": "intersects", "message": "Must be valid GeoJSON or WKT"}]}

Have some work in progress to verify formatting so this may be dealt with already. Also possible an edge-case of argument handling.

The client should ideally do some sanity checking (and communicating to the user in a more specific, actionable manner) before sending to the server.

handle 404 on scene product better

Currently, the response contains HTML and this gets shown to the user. The API makes no suggestion/promise of what content is returned on this error so perhaps:

• if html, extract some text from an element?
• if html, ignore the content (the error name will still be printed)

Allow filters=filters syntax in get_scenes_list

Because of the dots in most of the keyword names, get_scenes_list forces you to use this syntax:

filters={'cloud_cover.estimate.lt' : 0.1}
s  = client.get_scenes_list(**filters)

This would be nicer:

s = client.get_scenes_list(filters={'cloud_cover.estimate.lt' : 0.1})

support for workspaces

• list workspaces
• create workspace
• use workspaces in other operations