samapriya/clip-ship-planet-cli

Clip and Ship application to batch clip using Planet's Clip API

Over the last decade or more I have been working with all things geospatial. From building applications to teaching, training and acting as a translator at large for technology needs. A Google Developer Expert for Google Earth Engine and Senior Product Manager at MAXAR, I lead Developer Relations and champion open data access apart from working on core APIs and infrastructure.

I created the Awesome Google Earth Engine Community Catalog, a thriving data commons welcoming over half a million monthly visits from over 160+ countries. I leverage geospatial expertise as an affiliate Faculty at the University of Hawaiʻi at Mānoa and I am a Designated Campus Colleague at the University of Arizona. Passionate about community building, My research explores big data analysis and geospatial applications, while I advocate for science communication and empower researchers through collaborative platforms and speaking engagements.

---

⚡ Top Language Stats

📚 Blog posts

• How ClimateEngine.org and Awesome GEE Community Catalog are Expanding Open Geospatial Commons
• Behind the Scenes: The Role of NSF’s Jetstream2 in Building the Awesome GEE Community Catalog
• From Paper to Pixels- Rediscovering Historical USGS Topo Maps in the Google Earth Engine Community…
• Community-Enhanced: Google Earth Engine Community Catalog Upgrades
• Google Cloud Projects and Earth Engine Integration in Universities: A Practical Guide

Open Source Projects

• Google Earth Engine: awesome-gee-community-catalog | geeadd: Earth Engine Batch Asset Manager & addons | ee-appshot: Snapshot of Earth Engine Apps | geeup: Simple CLI for Earth Engine Uploads | cogee: Simple CLI for COG registration in GEE | Earth Engine dataset list |

• Open Oceans tools: pycoral: Simple CLI for Allen Coral Atlas | pyspotter: Simple CLI for SofarOcean API for spotters | pyaqua: Simple Command Line Interface Tool for Aqualink.org API | argofloats: Simple CLI for ArgoVis and Argofloats | gfw: Simple CLI for Global Fishing Watch Data

• Science platform tools : appeears: Simple CLI for NASA AppEEARS API | geneutils: NCBI Blast hit tools | pynative: Simple CLI for native-land.ca | ArcticDEM: Batch search and download tools

• Standalone tools : nsfsearch: NSF award search tool | spotifind: Spotify tool to generate playlists | imgurfy: Simple CLI for Imgur | pydrop: Simple client for digital ocean | ghome: Simple CLI for Google Home & Mini

If you find my work useful buy me a cup of tea to make these efforts more sustainable. Use the donate or sponsor buttons, I am happy to have you as a sponsor of my work

---

⚡ Recent GitHub Activity

1. 🗣 Commented on #43 in samapriya/awesome-gee-community-datasets
2. 🔒 Closed issue #43 in samapriya/awesome-gee-community-datasets
3. 🗣 Commented on #178 in samapriya/awesome-gee-community-datasets
4. 🗣 Commented on #270 in samapriya/awesome-gee-community-datasets
5. 🔒 Closed issue #270 in samapriya/awesome-gee-community-datasets

---

⚡ GitHub Stats

clip-ship-planet-cli's Issues

No Response to Clipping Request

Hi Samapriya,

When I ran the clipping request, there was no response (idling more than 20 min). Is that normal?
Here is the terminal result (Mac OS, Python 2.7):

$ sudo pclip activate --aoi "/Users/shengpan/Documents/PalnetClipShipCLI/Clip-Ship-Planet-CLI/aoi.json" --action "activate" --asset "PSScene4Band analytic"
Searching...
250 available images
20170915_071424_0f1b analytic PSScene4Band: already active
20170915_071423_0f1b analytic PSScene4Band: already active
20170928_071403_0f42 analytic PSScene4Band: already active
20170928_071402_0f42 analytic PSScene4Band: already active
20170926_071820_0e3a analytic PSScene4Band: already active
20170926_071819_0e3a analytic PSScene4Band: already active
20170905_071405_0f28 analytic PSScene4Band: already active
20170905_071404_0f28 analytic PSScene4Band: already active
20170926_071531_1031 analytic PSScene4Band: already active
20170925_071504_1018 analytic PSScene4Band: already active
20170925_071502_1018 analytic PSScene4Band: already active
20170924_071435_1023 analytic PSScene4Band: already active
20170909_071423_0f34 analytic PSScene4Band: already active
20170924_071343_0f3f analytic PSScene4Band: already active
20170824_071104_102c analytic PSScene4Band: already active
20170824_071103_102c analytic PSScene4Band: already active
20170923_071448_0f17 analytic PSScene4Band: already active
20170923_071447_0f17 analytic PSScene4Band: already active
20170922_071410_1034 analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170812_091152_104f
Could not activate - asset type 'analytic' not found for 20170909_090804_104d
Could not activate - asset type 'analytic' not found for 20170909_090803_104d
Could not activate - asset type 'analytic' not found for 20170903_090850_1020
Could not activate - asset type 'analytic' not found for 20170903_090851_1020
Could not activate - asset type 'analytic' not found for 20170822_091025_100d
Could not activate - asset type 'analytic' not found for 20170921_090633_0f53
20170920_071409_101f analytic PSScene4Band: already active
20170918_143138_1_0c46 analytic PSScene4Band: already active
20170919_071319_0f42 analytic PSScene4Band: already active
20170917_071138_1030 analytic PSScene4Band: already active
20170918_071451_0f52 analytic PSScene4Band: already active
20170918_071450_0f52 analytic PSScene4Band: already active
20170917_071322_1012 analytic PSScene4Band: already active
20170917_071320_1012 analytic PSScene4Band: already active
20170915_063355_0c0b analytic PSScene4Band: already active
20170915_063354_1_0c0b analytic PSScene4Band: already active
20170914_071343_0f17 analytic PSScene4Band: already active
20170913_071516_1044 analytic PSScene4Band: already active
20170913_071515_1044 analytic PSScene4Band: already active
20170911_071311_1003 analytic PSScene4Band: already active
20170911_071310_1003 analytic PSScene4Band: already active
20170910_071423_0f12 analytic PSScene4Band: already active
20170910_071422_0f12 analytic PSScene4Band: already active
20170908_071359_101e analytic PSScene4Band: already active
20170908_071358_101e analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170907_090735_0f21
20170630_071057_102d analytic PSScene4Band: already active
20170630_071056_102d analytic PSScene4Band: already active
20170904_071316_1044 analytic PSScene4Band: already active
20170904_071317_1044 analytic PSScene4Band: already active
20170902_071413_100e analytic PSScene4Band: already active
20170902_071412_100e analytic PSScene4Band: already active
20170831_071433_0f25 analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170825_090858_0f2d
Could not activate - asset type 'analytic' not found for 20170825_090857_0f2d
20170827_071153_103d analytic PSScene4Band: already active
20170830_071251_1041 analytic PSScene4Band: already active
20170830_071250_1041 analytic PSScene4Band: already active
20170827_071804_0e1f analytic PSScene4Band: already active
20170827_071802_0e1f analytic PSScene4Band: already active
20170826_071449_0e26 analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170818_091119_0f44
Could not activate - asset type 'analytic' not found for 20170818_091118_0f44
20170824_071205_1002 analytic PSScene4Band: already active
20170824_071204_1002 analytic PSScene4Band: already active
20170824_071205_100e analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170821_091101_0f37
20170822_071313_103c analytic PSScene4Band: already active
20170822_071312_103c analytic PSScene4Band: already active
20170821_071352_1027 analytic PSScene4Band: already active
20170821_071351_1027 analytic PSScene4Band: already active
20170820_071218_1024 analytic PSScene4Band: already active
20170820_071219_1024 analytic PSScene4Band: already active
20170819_071123_1021 analytic PSScene4Band: already active
20170819_071122_1021 analytic PSScene4Band: already active
20170819_071051_100c analytic PSScene4Band: already active
20170819_071050_100c analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170818_091024_0f3c
20170818_071152_0f31 analytic PSScene4Band: already active
20170816_071119_101d analytic PSScene4Band: already active
20170815_071302_102f analytic PSScene4Band: already active
20170815_071301_102f analytic PSScene4Band: already active
Could not activate - asset type 'analytic' not found for 20170815_091132_0f53
Could not activate - asset type 'analytic' not found for 20170815_091133_0f53
20170814_071245_1023 analytic PSScene4Band: already active
20170814_071244_1023 analytic PSScene4Band: already active
20170813_071153_0f43 analytic PSScene4Band: already active
20170813_071152_0f43 analytic PSScene4Band: already active
20170812_071244_101a analytic PSScene4Band: already active
20170727_120429_0c46 analytic PSScene4Band: already active
20170727_120428_0c46 analytic PSScene4Band: already active
20170729_071105_1041 analytic PSScene4Band: already active
20170729_071104_1041 analytic PSScene4Band: already active
20170730_071258_0f25 analytic PSScene4Band: already active
20170730_071832_0e1f analytic PSScene4Band: already active
20170731_071113_1014 analytic PSScene4Band: already active
20170731_071112_1014 analytic PSScene4Band: already active
20170731_071124_1022 analytic PSScene4Band: already active
20170731_071123_1022 analytic PSScene4Band: already active
20170808_071741_0e0f analytic PSScene4Band: already active
20170808_071740_0e0f analytic PSScene4Band: already active
20170807_071133_1012 analytic PSScene4Band: already active
20170804_071330_0f28 analytic PSScene4Band: already active
20170803_071155_1044 analytic PSScene4Band: already active
20170803_071154_1044 analytic PSScene4Band: already active
20170727_071251_1008 analytic PSScene4Band: already active
20170725_071018_1006 analytic PSScene4Band: already active
20170725_071712_0e0f analytic PSScene4Band: already active
20170725_071218_1013 analytic PSScene4Band: already active
20170725_071217_1013 analytic PSScene4Band: already active
20170723_071105_102f analytic PSScene4Band: already active
20170722_071148_1023 analytic PSScene4Band: already active
20170723_071726_0e0d analytic PSScene4Band: already active
20170722_071148_1009 analytic PSScene4Band: already active
20170722_071147_1009 analytic PSScene4Band: already active
20170721_071147_101b analytic PSScene4Band: already active
20170720_071058_1027 analytic PSScene4Band: already active
20170720_071057_1027 analytic PSScene4Band: already active
20170718_071031_1001 analytic PSScene4Band: already active
20170718_071030_1001 analytic PSScene4Band: already active
20170711_075108_0c19 analytic PSScene4Band: already active
20170715_064011_1_0c75 analytic PSScene4Band: already active
20170715_064011_0c75 analytic PSScene4Band: already active
20170717_071216_0f41 analytic PSScene4Band: already active
20170716_071742_0e1f analytic PSScene4Band: already active
20170715_071053_1019 analytic PSScene4Band: already active
20170712_071033_0f17 analytic PSScene4Band: already active
20170712_071034_0f17 analytic PSScene4Band: already active
20170711_071129_1044 analytic PSScene4Band: already active
20170711_071151_0f4e analytic PSScene4Band: already active
20170708_071111_1032 analytic PSScene4Band: already active
20170708_070939_1022 analytic PSScene4Band: already active
20170708_070940_1022 analytic PSScene4Band: already active
20170707_070919_0f34 analytic PSScene4Band: already active
20170707_070918_0f34 analytic PSScene4Band: already active
20170706_071003_101e analytic PSScene4Band: already active
20170706_071025_1041 analytic PSScene4Band: already active
20170706_071024_1041 analytic PSScene4Band: already active
20170703_071129_0f31 analytic PSScene4Band: already active
20170702_071559_0e1f analytic PSScene4Band: already active
20170702_071558_0e1f analytic PSScene4Band: already active
20170702_071058_1013 analytic PSScene4Band: already active
20170702_071057_1013 analytic PSScene4Band: already active
20170701_071106_101d analytic PSScene4Band: already active
20170701_071105_101d analytic PSScene4Band: already active
20170701_070908_1036 analytic PSScene4Band: already active
20170624_070954_0f42 analytic PSScene4Band: already active
20170624_070956_0f15 analytic PSScene4Band: already active
20170624_070955_0f15 analytic PSScene4Band: already active
20170623_071528_0e20 analytic PSScene4Band: already active
20170622_070926_1019 analytic PSScene4Band: already active
20170622_070948_1012 analytic PSScene4Band: already active
20170621_070940_1030 analytic PSScene4Band: already active
20170618_071017_0f4e analytic PSScene4Band: already active
20170618_071016_0f4e analytic PSScene4Band: already active
20170615_070922_1023 analytic PSScene4Band: already active
20170615_070923_1023 analytic PSScene4Band: already active
20170614_071455_0e14 analytic PSScene4Band: already active
20170614_071454_0e14 analytic PSScene4Band: already active
20170614_070904_0f10 analytic PSScene4Band: already active
20170614_070903_0f10 analytic PSScene4Band: already active
20170614_070936_0f25 analytic PSScene4Band: already active
20170614_070935_0f25 analytic PSScene4Band: already active
20170526_070839_1004 analytic PSScene4Band: already active
20170526_070838_1004 analytic PSScene4Band: already active
20170610_070855_1033 analytic PSScene4Band: already active
20170610_070856_1033 analytic PSScene4Band: already active
20170609_072722_0d05 analytic PSScene4Band: already active
20170609_072721_0d05 analytic PSScene4Band: already active
20170517_071421_0e19 analytic PSScene4Band: already active
20170528_123115_0c59 analytic PSScene4Band: already active
20170528_123114_0c59 analytic PSScene4Band: already active
20170521_044819_1_0c37 analytic PSScene4Band: already active
20170517_071422_0e19 analytic PSScene4Band: already active
20170603_070844_1007 analytic PSScene4Band: already active
20170603_070843_1007 analytic PSScene4Band: already active
20170512_070753_1004 analytic PSScene4Band: already active
20170512_070752_1004 analytic PSScene4Band: already active
20170507_070748_101a analytic PSScene4Band: already active
20170507_070747_101a analytic PSScene4Band: already active
20170521_053802_0c54 analytic PSScene4Band: already active
20170516_064442_1_0c27 analytic PSScene4Band: already active
20170510_091822_1_0c43 analytic PSScene4Band: already active
20170510_091822_0c43 analytic PSScene4Band: already active
20170520_070816_1005 analytic PSScene4Band: already active
20170517_070750_0f34 analytic PSScene4Band: already active
20170517_062810_0c46 analytic PSScene4Band: already active
20170515_071444_0e2f analytic PSScene4Band: already active
20170509_070717_1029 analytic PSScene4Band: already active
20170502_130148_0c38 analytic PSScene4Band: already active
20170504_070719_100c analytic PSScene4Band: already active
20170503_070706_0f34 analytic PSScene4Band: already active
20170502_070738_1019 analytic PSScene4Band: already active
20170420_070641_1029 analytic PSScene4Band: already active
20170423_070724_1038 analytic PSScene4Band: already active
20170424_070642_0f28 analytic PSScene4Band: already active
20170424_070641_0f28 analytic PSScene4Band: already active
20170418_070711_1019 analytic PSScene4Band: already active
20170420_071304_0e0e analytic PSScene4Band: already active
20170419_070705_0f25 analytic PSScene4Band: already active
20170419_070704_0f25 analytic PSScene4Band: already active
20170415_070647_0f35 analytic PSScene4Band: already active
20170410_070638_1010 analytic PSScene4Band: already active
20170414_071256_0e30 analytic PSScene4Band: already active
20170414_071255_0e30 analytic PSScene4Band: already active
20170331_113547_0c42 analytic PSScene4Band: already active
20170331_113546_0c42 analytic PSScene4Band: already active
20170331_113746_0c22 analytic PSScene4Band: already active
20170331_113745_0c22 analytic PSScene4Band: already active
20170327_131744_0c81 analytic PSScene4Band: already active
20170318_063622_0c65 analytic PSScene4Band: already active
20170318_063621_0c65 analytic PSScene4Band: already active
20170317_073830_0c54 analytic PSScene4Band: already active
20170305_122125_0c45 analytic PSScene4Band: already active
20170316_070458_1018 analytic PSScene4Band: already active
20170316_070457_1018 analytic PSScene4Band: already active
20170124_141339_0c42 analytic PSScene4Band: already active
20170124_141338_0c42 analytic PSScene4Band: already active
20170314_071249_0e0f analytic PSScene4Band: already active
20170307_071301_0e0d analytic PSScene4Band: already active
20170223_071216_0e1f analytic PSScene4Band: already active
20170129_071144_0e0f analytic PSScene4Band: already active
20170129_071143_0e0f analytic PSScene4Band: already active
20170127_071225_0e3a analytic PSScene4Band: already active
20170106_071019_0e19 analytic PSScene4Band: already active
20170106_071018_0e19 analytic PSScene4Band: already active
20161111_095702_0c45 analytic PSScene4Band: already active
20161111_095701_0c45 analytic PSScene4Band: already active
20161109_070915_0e3a analytic PSScene4Band: already active
20161109_070914_0e3a analytic PSScene4Band: already active
20161106_114054_1_0c81 analytic PSScene4Band: already active
20161105_070902_0e0f analytic PSScene4Band: already active
20161214_070011_0c59 analytic PSScene4Band: already active
20161202_070940_0e19 analytic PSScene4Band: already active
20161127_133931_0c81 analytic PSScene4Band: already active
20161126_142014_0c54 analytic PSScene4Band: already active
20161126_142013_0c54 analytic PSScene4Band: already active
20161114_070908_0e3a analytic PSScene4Band: already active
20161114_070909_0e3a analytic PSScene4Band: already active
20161102_070816_0e30 analytic PSScene4Band: already active
20161102_070814_0e30 analytic PSScene4Band: already active
20161021_070847_0e1f analytic PSScene4Band: already active
20161021_070848_0e1f analytic PSScene4Band: already active
20161015_070835_0e3a analytic PSScene4Band: already active
20161015_070834_0e3a analytic PSScene4Band: already active
20161010_070753_0e3a analytic PSScene4Band: already active
20161010_070751_0e3a analytic PSScene4Band: already active
20161007_070718_0e19 analytic PSScene4Band: already active
20161002_120755_0c19 analytic PSScene4Band: already active
20161002_120756_0c19 analytic PSScene4Band: already active
Requested activation for 0 of 250 items
(PlanetPrjs27) user-8a51aa:Clip-Ship-Planet-CLI shengpan $ sudo pclip jsonc --path "/Users/shengpan/Documents/PalnetClipShipCLI/Clip-Ship-Planet-CLI/aoi.json" --item "PSScene4Band" --asset "analytic"

long time waiting after this

Add SkySatScene & associated asset-types to CLI

pclip: error: unrecognized arguments: --asst SkySatScene ortho_visual

I would like to request the addition of SkySatScene as an item-type option as well as the associated asset-types. In particular ortho_visual & ortho_pansharpened for my use case. Thank you!

NameError: name 'sys' is not defined

I installed the latest version, on windows 10, successfully did the python setup.py install ..no error messages, but when I typed pclip -h to test. I got error messages saying "NameError: name 'sys' is not defined" . What have I done wrong?

Error in Saving Planet API Key

Hi Samapriya,

Have you tested the command "pclip planetkey"? It keeps telling me "Sorry, try again" when I copy-pasted my Planet API Key. I have double-checked my API Key in my account and it should be right.

clip_download seems to get stuck at lines 34-41

I'm downloading some long time series of small fields (3 years) and downloadclips seems to get stuck at the section indicated below in clip_download.py:

import requests,json,re,csv,os,subprocess,urllib2,getpass,time,progressbar,pdb
from pprint import pprint
from os.path import expanduser
from urllib2 import Request, urlopen
from planet.api.utils import read_planet_json
os.chdir(os.path.dirname(os.path.realpath(__file__)))
planethome=os.path.dirname(os.path.realpath(__file__))
try:
        PL_API_KEY = read_planet_json()['key']
except:
        subprocess.call('planet init',shell=True)
CAS_URL='https://api.planet.com/compute/ops/clips/v1/'
headers = {'Content-Type': 'application/json',}
if not os.path.exists(os.path.join(planethome,'urllist.csv')):
        with open(os.path.join(planethome,'urllist.csv'),'w') as completed:
            writer=csv.writer(completed,delimiter=',',lineterminator='\n')
with open(os.path.join(planethome,"urllist.csv")) as f:
    csv_f = csv.reader(f)
    value = len(list(csv_f))
def downloadclips(filepath=None):
    with open(os.path.join(planethome,"urllist.csv")) as f:
        csv_f = csv.reader(f)
        for i,row in enumerate(csv_f):
            URL=str(row).replace("'","").replace("[","").replace("]","")
            downlink = requests.get(url=URL, auth=(PL_API_KEY, ''))
            if downlink.status_code==200:
                content=downlink.json()
                item_id=(content['targets'][0]['item_id'])
                item_typ=(content['targets'][0]['item_type'])
                asset_typ=(content['targets'][0]['asset_type'])
                try:
                    if content['state']=='running':
                        ### THE CODE GETS STUCK IN THE BELOW WHILE LOOP ###
                        while content['state']=='running':
                            bar = progressbar.ProgressBar()
                            for i in bar(range(60)):
                                time.sleep(1)
                            downlink = requests.get(url=URL, auth=(PL_API_KEY, ''))
                            if downlink.status_code==200:
                                content=downlink.json()
                            try_number = try_number + 1
                       ### THE CODE GETS STUCK IN THE ABOVE WHILE LOOP ###
                    elif content['state']=='succeeded':
                        filelink = urllib2.urlopen(str(content['_links']['results'][0]))
                        filename=item_id+"_"+item_typ+"_"+asset_typ+".zip"
                        ov=os.path.join(filepath,filename)
                        if not os.path.exists(ov):
                            with open(ov, "wb") as code:
                                code.write(filelink.read())
                            print("Downloading: "+str(i+1)+" of "+str(value)+" "+(item_id+"_"+item_typ+"_"+asset_typ))
                        else:
                            print("asset exists..Skipping "+str(filename))

                    else:
                        print("Clip Function still " +content['state']+" "+str(filename))
                except Exception as e:
                    print e
            else:
                print("Got Download Return Error Code: "+str(downlink.status_code))

I'm a python newbie, but I can't see how it gets from that while loop to the bit where it looks like it's actually doing the downloading: filelink = urllib2.urlopen(str(content['_links']['results'][0])) and code.write(filelink.read()).

Get the tool to check if assets exist before activating, creating the idlist, and clipping?

Currently, the tool only checks if the asset exists at the download stage. Is it feasible to get the tool to also check if assets exist before activating, creating the idlist, and clipping? This could save a huge amount of time when going back to fill in gaps of failed downloads.

Install

Trying to install it in windows and i get the following error

Traceback (most recent call last):
File "setup.py", line 27, in
long_description=open('README.rst').read(),
File "C:\Users\User\AppData\Local\Programs\Python\Python36\lib\encodings\cp1253.py", line 23, in decode
return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9c in position 10550: character maps to

Any ideas?