crehak1/arcgis-sentinel2-download-tools

ArcGIS Python Toolbox for Search & Download of Sentinel-2 data (L1C, L2A) via DHuS (Copernicus Open Access Hub a.k.a. SciHub, or CODE-DE).

Purpose

The purpose is to provide assistance to occasional users to deal with Sentinel data in a local ArcGIS Desktop environment by means of two Python Toolbox tools and common ArcGIS Desktop experience:

1. Search Data Hub Services' (DHuS) product catalog for Sentinel-2 products (L1C, or L2A where available) according to given criteria (in particular spatiotemporal constraints and cloud cover limit).
   The search results are stored in a local product catalog (raster catalog) acting as a controllable cache (for product metadata and previews).
   
2. Interactively browse metadata (attribute table) and product previews (by selecting product records); mark desired product records for download.
3. For each Marked entry in the local product catalog, Download the respective raster data package.
   

Upon download success, the respective raster datasets are displayed in ArcMap.

Characteristics

• Level-2A (L2A) products, available from ESA's Copernicus Open Access Hub (SciHub DHuS) since begin of May 2017, are taken into account as a new option, but ArcMap does not yet support them in terms of a built-in raster product. As a substitute, the Download tool displays a L2A product by a Group Layer until further notice, composed of confidence images for cloud (CLD) and snow/ice (SNW), a natural color composite (TCI), and a scene classification image (SCL), along with appropriate symbology:
  
• Irrespective of the existence of a specific product in the specified local raster directory, the Download tool generates the corresponding output layer(s), even though all redundant download actions have been skipped. This might be useful when you simply want to recreate those layers incl. appropriate symbology.
• The python interpreter automatically takes proxy server settings into account, either based on respective Internet Explorer settings, or based on the two environment variables https_proxy + http_proxy (incl. optional Basic Authorization. See the proxy-example*.* files within the doc directory).
• CODE-DE support (since March 2017): The German mirror site provides DHuS similary to Copernicus Open Access Hub (SciHub site). The Toolbox tools do now allow for choosing CODE-DE as an alternative DHuS site (in addition to the mainstream DHuS at SciHub site).
  📓 Note: It may happen that certain data sets are missing at CODE-DE site. If in doubt, rerun the respective tool against the reference DHuS at SciHub site.
• ArcGIS 10.5.1 now supports the newer version (>=14) of the Products Specification Document (PSD) in terms of a built-in raster product. ArcGIS versions lower than 10.5.1 lack such support. As a substitute, the natural color composite (TCI) is then used for the Download tool output parameter.
• A multi-tile package is represented by multiple records in the search results table (raster catalog) rather than by one single product record. This way each tile preview can be examined individually. When it comes to downloading such a multi-tile product, the Download tool treats those multiple records as a single entity, i.e. a single (full) product download will be performed regardless of how many tiles are Marked.
  Over time, this kind of quirk loses its relevance because newer products (those provided since the end of September 2016) are provided solely as single-tile packages.
• Contrary to the previous point: When using the "Image selection" mode of the Download tool, the download of product images is performed with Marked tiles only, i.e. non-marked tiles are actually ignored even if they are part of a multi-tile package.

Prerequisites

• Valid login credentials for DHuS at either one of the following
  • Copernicus Open Access Hub: https://scihub.copernicus.eu/dhus/#/self-registration
  • CODE-DE site: https://code-de.org/dhus/#/self-registration
• Tested with ArcMap/ArcCatalog version 10.4.1, 10.5, 10.5.1 (not suitable for ArcGIS Pro until further notice).
• Regarding ArcMap 10.4.1:
  • ArcGIS 10.4.1 Raster Patch.
  • On affected systems: ArcGIS Runtime Error R6034 Patch.

Getting Started

• Download ZIP and extract its content to a local directory that can be reached by an ArcCatalog Folder Connection.
  Make sure that the original file structure is preserved (relative paths); all referenced files have to be properly placed with respect to the main Toolbox files (.pyt, *.xml), by name sensub.py and all *.lyr files within their respective subdirectory.
  📓 Note: Do not simply drag and drop the Toolbox icon to a desired ArcCatalog Folder Connection (e.g. "My Toolboxes"), because by doing so ArcCatalog copies only the Toolbox .pyt file in conjunction with its belonging *.xml help files but leaves out all other dependent files!
• Before using the tools, it is highly advised to read the respective Item Description of the Toolbox and of each tool in advance (see respective context menu in ArcMap), particularly the Usage of each tool (also reachable from each Tool Help). Amongst others, the Usage of the Search tool introductorily explains some general ArcMap settings that have to be carried out in advance.
  When running the parameter form of a particular tool, consult the respective Parameter Explanation shown in the side panel (button "Show Help >>" opens the side panel).