umr-lops / cdsodatacli Goto Github PK
View Code? Open in Web Editor NEWodata client for Copernicus Data Space catalog
Home Page: https://cerweb.ifremer.fr/datarmor/doc_sphinx/cdsodatacli/index.html
License: MIT License
cdsodatacli's People
Contributors
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
Stargazers
Watchers
cdsodatacli's Issues
Using mode="multi" triggers KeyError
🐛 Bug Report
When querying using a gdf, and using mode="multi".
It is systematic, for each fetch_data call.
For the same queries using mode="seq" this doesn't happen.
Traceback (most recent call last):
File "/home1/datahome/oarcher/storm_watch/./bt2sar_new.py", line 216, in <module>
safes = cdsodatacli.query.fetch_data(gdf, timedelta_slice=datetime.timedelta(weeks=1), min_sea_percent=15, mode="m
ulti")
File "/home1/datahome/oarcher/storm_watch/conda_bt2sar_new/lib/python3.10/site-packages/cdsodatacli/query.py", line
169, in fetch_data
data_dedup = remove_duplicates(safes_ori=collected_data)
File "/home1/datahome/oarcher/storm_watch/conda_bt2sar_new/lib/python3.10/site-packages/cdsodatacli/query.py", line
683, in remove_duplicates
safes_sort = safes_ori.sort_values("ModificationDate", ascending=False)
File "/home1/datahome/oarcher/storm_watch/conda_bt2sar_new/lib/python3.10/site-packages/pandas/util/_decorators.py", line 331, in wrapper
return func(*args, **kwargs)
File "/home1/datahome/oarcher/storm_watch/conda_bt2sar_new/lib/python3.10/site-packages/pandas/core/frame.py", line 6912, in sort_values
k = self._get_label_or_level_values(by, axis=axis)
File "/home1/datahome/oarcher/storm_watch/conda_bt2sar_new/lib/python3.10/site-packages/pandas/core/generic.py", lin
e 1850, in _get_label_or_level_values
raise KeyError(key)
KeyError: 'ModificationDate'
handle round-robin multiple CDSE account
- create new accounts
- dev a tool to check that quotas and CDSE limitations are respected
add a wrapper method to let a user define a CDSE query without having to create the GeoDataFrame explicitely
- clean fetch_data arguments
- wrapper for
fetch_datamethod
add a permission change 775 on the .zip file download
test unitaire
Retrieve adjacent safes
Is it possible to add a parameter to also match adjacent SAFES (for example if adjacent=2, we match the 2 closest adjacent SAFES, for each SAFE) ?
OSError: [Errno 101] Network is unreachable
🐛 Bug Report
a query with a single polygon on new caledonia from 2014 up to now crashes.
traceback
ConnectionError: HTTPSConnectionPool(host='catalogue.dataspace.copernicus.eu', port=443): Max retries exceeded with url: /odata/v1/Products?$filter=OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((162%20-19,%20169%20-19,%20169%20-23,%20162%20-23,%20162%20-19))')%20and%20Collection/Name%20eq%20'SENTINEL-1'%20and%20Attributes/OData.CSC.StringAttribute/any(att:att/Name%20eq%20'operationalMode'%20and%20att/OData.CSC.StringAttribute/Value%20eq%20'IW')%20and%20Attributes/OData.CSC.StringAttribute/any(att:att/Name%20eq%20'productType'%20and%20att/OData.CSC.StringAttribute/Value%20eq%20'SLC')%20and%20ContentDate/Start%20gt%202014-04-06T00:00:00.000Z%20and%20ContentDate/Start%20lt%202014-04-13T00:00:00.000Z&$top=1000&$expand=Attributes (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2aab241a1bd0>: Failed to establish a new connection: [Errno 101] Network is unreachable'))to reproduce:
sta = datetime.datetime(2014,4,6)
sto = datetime.datetime.today()
poly = shapely.wkt.loads("POLYGON ((162 -19, 169 -19, 169 -23, 162 -23, 162 -19))")
gdf = gpd.GeoDataFrame({
"start_datetime" : [ sta ],
"end_datetime" : [ sto],
"geometry" : [ poly],
"collection" : [ "SENTINEL-1"],
"name" : [ None],
"sensormode" : [ 'IW'],
"producttype" : [ 'SLC'],
"Attributes" : [ None],
})
collected_data_norm = cdsodatacli.query.fetch_data(gdf, min_sea_percent=20)use logging instead of print
handle the different code status return by OData API
OSError: [Errno 18] Invalid cross-device link
traceback
os.rename(src, real_dst)
OSError: [Errno 18] Invalid cross-device link: '/ho....atch/satwave/sentinel1_cdse_pre_spool/S1A_WV_SLC__1SSV_20240225T013147_20240225T015559_052708_0660A3_F6B6.SAFE
.zip.tmp' -> '/ho.....ublic/project/mpc-sentinel1/workspace/spool_datarmor/S1A_WV_SLC__1SSV_20240225T013147_20240225T015559_052708_0660A3_F6B6.SAFE.zip'it is recommended to use shutil.move for such operations.
improve the blacklist system
Currently when 4 errors are through in row for a given CDSE account, then the accound is black listed for the rest of the processing.
It would be clever to distinguish between status code/meaning of errors to avoid account starvation due to ChunkedEncodingError for instance.
error on WV SLC product download (incomplet read)
this issue crash (randomly?) when downloading product for which donwload is quite long (IW or WV SLC for instance).
product in error:
8c695363-f650-434c-b465-8364e59f7481,S1A_WV_SLC__1SSV_20231201T064122_20231201T071521_051457_0635DA_ED84.SAFE
A traceback
01/12/2023 12:03:33 INFO download.py(491) start download : S1A_WV_SLC__1SSV_20231201T064122_20231201T071521_051457_0635DA_ED84.SAFE
CDSE download defaultdict(<class 'int'>, {'total_product_to_download': 9648, 'product_absent_from_local_disks': 329, 'archived_product': 9319}): 0%| | 0/20 [24:41<?, ?it/s]
Traceback (most recent call last):
File "/opt/conda-envs/dev/lib/python3.10/site-packages/urllib3/response.py", line 710, in _error_catcher
yield
File "/opt/conda-envs/dev/lib/python3.10/site-packages/urllib3/response.py", line 835, in _raw_read
raise IncompleteRead(self._fp_bytes_read, self.length_remaining)
urllib3.exceptions.IncompleteRead: IncompleteRead(4575162672 bytes read, 8620036116 more expected)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda-envs/dev/lib/python3.10/site-packages/requests/models.py", line 816, in generate
yield from self.raw.stream(chunk_size, decode_content=True)
File "/opt/conda-envs/dev/lib/python3.10/site-packages/urllib3/response.py", line 940, in stream
data = self.read(amt=amt, decode_content=decode_content)
File "/opt/conda-envs/dev/lib/python3.10/site-packages/urllib3/response.py", line 911, in read
data = self._raw_read(amt)
File "/opt/conda-envs/dev/lib/python3.10/site-packages/urllib3/response.py", line 813, in _raw_read
with self._error_catcher():
File "/opt/conda-envs/dev/lib/python3.10/contextlib.py", line 153, in __exit__
self.gen.throw(typ, value, traceback)
File "/opt/conda-envs/dev/lib/python3.10/site-packages/urllib3/response.py", line 727, in _error_catcher
raise ProtocolError(f"Connection broken: {e!r}", e) from e
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(4575162672 bytes read, 8620036116 more expected)', IncompleteRead(4575162672 bytes read, 8620036116 more expected))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home1/datahome/agrouaze/sources/git/cdsodatacli/tests_metiers/test_download_WV.py", line 66, in <module>
dfout = download_list_product_sequential(
File "/home1/datahome/agrouaze/sources/git/cdsodatacli/cdsodatacli/download.py", line 529, in download_list_product_sequential
) = CDS_Odata_download_one_product_v2(
File "/home1/datahome/agrouaze/sources/git/cdsodatacli/cdsodatacli/download.py", line 110, in CDS_Odata_download_one_product_v2
for chunk in response.iter_content(chunk_size=8192):
File "/opt/conda-envs/dev/lib/python3.10/site-packages/requests/models.py", line 818, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(4575162672 bytes read, 8620036116 more expected)', IncompleteRead(4575162672 bytes read, 8620036116 more expected))it is not clear whether it is an issue coming from cdsodatacli or from the OData API, a ticket is open in support system CDSE:
sphinx documentation
Add cache feature
add retry mechanism
When a query is on failure -> retry 1 or 2 times.
Json parsing error
🐛 Bug Report
41%|████ | 50/122 [00:21<00:29, 2.44it/s]ERROR:root:impossible to get data from CDSfor query: https://catalogue
.dataspace.copernicus.eu/odata/v1/Products?$filter=OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((127.25 31.23
333333333333, 127.2432586173411 31.09610933687195, 127.2230993925645 30.96020688251075, 127.1897164700251 30.826934785
17709, 127.1434313455158 30.69757652802221, 127.0846897700877 30.57337790177694, 127.0140574572236 30.45553500710589,
126.9322146347078 30.34518273550423, 126.8399494936612 30.24338383967217, 126.7381505978291 30.1511186986255, 126.6277
983262274 30.06927587610977, 126.5099554315564 29.99864356324564, 126.3857568053111 29.93990198781754, 126.25639854815
62 29.89361686330824, 126.1231264508226 29.86023394076881, 125.9872239964614 29.84007471599226, 125.85 29.833333333333
34, 125.7127760035386 29.84007471599226, 125.5768735491774 29.86023394076881, 125.4436014518437 29.89361686330824, 125
.3142431946889 29.93990198781753, 125.1900445684436 29.99864356324564, 125.0722016737726 30.06927587610977, 124.961849
4021709 30.1511186986255, 124.8600505063388 30.24338383967217, 124.7677853652922 30.34518273550423, 124.6859425427764
30.45553500710589, 124.6153102299123 30.57337790177694, 124.5565686544842 30.69757652802221, 124.5102835299749 30.8269
3478517708, 124.4769006074355 30.96020688251075, 124.4567413826589 31.09610933687195, 124.45 31.23333333333333, 124.45
67413826589 31.37055732979472, 124.4769006074355 31.50645978415591, 124.5102835299749 31.63973188148958, 124.556568654
4842 31.76909013864446, 124.6153102299123 31.89328876488973, 124.6859425427764 32.01113165956077, 124.7677853652922 32
.12148393116244, 124.8600505063388 32.2232828269945, 124.9618494021709 32.31554796804117, 125.0722016737726 32.3973907
905569, 125.1900445684436 32.46802310342103, 125.3142431946889 32.52676467884913, 125.4436014518437 32.57304980335843,
125.5768735491774 32.60643272589785, 125.7127760035386 32.62659195067441, 125.85 32.63333333333333, 125.9872239964614
32.62659195067441, 126.1231264508226 32.60643272589786, 126.2563985481562 32.57304980335843, 126.3857568053111 32.526
76467884914, 126.5099554315564 32.46802310342103, 126.6277983262274 32.3973907905569, 126.7381505978291 32.31554796804
117, 126.8399494936612 32.22328282699451, 126.9322146347078 32.12148393116244, 127.0140574572236 32.01113165956078, 12
7.0846897700877 31.89328876488974, 127.1434313455158 31.76909013864447, 127.1897164700251 31.63973188148959, 127.22309
93925645 31.50645978415593, 127.2432586173411 31.37055732979473, 127.25 31.23333333333334, 127.25 31.23333333333333))'
) and Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and a
tt/OData.CSC.StringAttribute/Value eq 'GRD') and ContentDate/Start gt 2022-09-05T06:30:00.000Z and ContentDate/Start l
t 2022-09-05T07:30:00.000Z&$top=1000&$expand=Attributes: Traceback (most recent call last):
File "/home1/datahome/oarcher/storm_watch/conda3/lib/python3.8/site-packages/cdsodatacli/query.py", line 500, in fet
ch_one_url
json_data = requests.get(url).json()
File "/home1/datahome/oarcher/storm_watch/conda3/lib/python3.8/site-packages/requests/models.py", line 898, in json
return complexjson.loads(self.text, **kwargs)
File "/home1/datahome/oarcher/storm_watch/conda3/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/home1/datahome/oarcher/storm_watch/conda3/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home1/datahome/oarcher/storm_watch/conda3/lib/python3.8/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)🔬 How To Reproduce
Il semble que ça ne se reproduise pas systématiquement du tout (même avec le même geodataframe en entrée). Je me demande si c'est pas lié à du rate-limiting, car on dirait que ça n'arrive que lorsque j'effectue plusieurs requêtes + ou - d'affilée.
Il faudrait que je chope le contenu du json en réponse...
Une première chose à faire dans le code de cdsodatacli, c'est d'afficher le contenu que renvoie le site web en cas derreur, avant de parser avec json.
J'essaye de faire ça de mon côté.
Environment
conda list
_libgcc_mutex 0.1 main.conda
argon2-cffi 20.1.0 py38h1e0a361_1 conda-forge
async_generator 1.10 py_0 conda-forge
atcf 0.0.4.dev8+g3ed850d <pip>
attrs 20.2.0 pyh9f0ad1d_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.1 py_0 conda-forge
bleach 3.2.1 pyh9f0ad1d_0 conda-forge
boost-cpp 1.74.0 h9359b55_0 conda-forge
brotlipy 0.7.0 py38h1e0a361_1000 conda-forge
bzip2 1.0.8 h516909a_3 conda-forge
c-ares 1.16.1 h516909a_3 conda-forge
ca-certificates 2020.6.20 hecda079_0 conda-forge
cairo 1.16.0 h3fc0475_1005 conda-forge
cdsodatacli 2023.12.19 <pip>
certifi 2020.6.20 py38h32f6830_0 conda-forge
cffi 1.14.3 py38h5bae8af_0 conda-forge
cfitsio 3.470 hce51eda_6 conda-forge
chardet 3.0.4 py38h32f6830_1007 conda-forge
click 8.1.7 <pip>
click 7.1.2 pyh9f0ad1d_0 conda-forge
click-plugins 1.1.1 py_0 conda-forge
cligj 0.5.0 py_0 conda-forge
cryptography 3.1 py38h766eaa4_0 conda-forge
curl 7.71.1 he644dc0_6 conda-forge
cycler 0.10.0 py_2 conda-forge
dbus 1.13.6 he372182_0 conda-forge
decorator 4.4.2 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
descartes 1.1.0 py_4 conda-forge
entrypoints 0.3 py38h32f6830_1001 conda-forge
expat 2.2.9 he1b5a44_2 conda-forge
fiona 1.9.5 <pip>
fiona 1.8.17 py38h676c6b2_0 conda-forge
fontconfig 2.13.1 h1056068_1002 conda-forge
freetype 2.10.2 he06d7ca_0 conda-forge
freexl 1.0.5 h516909a_1002 conda-forge
future 0.18.2 <pip>
gdal 3.1.2 py38hb61cb63_1 conda-forge
geo-shapely 0.0.6.dev7+g913bbce <pip>
geopandas 0.8.1 py_0 conda-forge
geopandas 0.13.2 <pip>
geopandas-coloc 0.0.4.dev2+g43cb5f3 <pip>
geos 3.8.1 he1b5a44_0 conda-forge
geotiff 1.6.0 ha04d9d0_1 conda-forge
gettext 0.19.8.1 hc5be6a0_1002 conda-forge
giflib 5.2.1 h516909a_2 conda-forge
glib 2.66.0 h0dae87d_0 conda-forge
gst-plugins-base 1.14.5 h0935bb2_2 conda-forge
gstreamer 1.14.5 h36ae1b5_2 conda-forge
hdf4 4.2.13 hf30be14_1003 conda-forge
hdf5 1.10.6 nompi_h3c11f04_101 conda-forge
html2text 2020.1.16 <pip>
icu 67.1 he1b5a44_0 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
importlib-metadata 1.7.0 py38h32f6830_0 conda-forge
importlib_metadata 1.7.0 0 conda-forge
ipykernel 5.3.4 py38h23f93f0_0 conda-forge
ipython 7.18.1 py38h1cdfbd6_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
ipywidgets 7.5.1 pyh9f0ad1d_1 conda-forge
jedi 0.15.2 py38_0 conda-forge
jinja2 2.11.2 pyh9f0ad1d_0 conda-forge
jpeg 9d h516909a_0 conda-forge
json-c 0.13.1 hbfbb72e_1002 conda-forge
jsonschema 3.2.0 py38h32f6830_1 conda-forge
jupyter 1.0.0 py_2 conda-forge
jupyter_client 6.1.7 py_0 conda-forge
jupyter_console 6.2.0 py_0 conda-forge
jupyter_core 4.6.3 py38h32f6830_1 conda-forge
jupyterlab_pygments 0.1.1 pyh9f0ad1d_0 conda-forge
kealib 1.4.13 h33137a7_1 conda-forge
kiwisolver 1.2.0 py38hbf85e49_0 conda-forge
krb5 1.17.1 hfafb76e_3 conda-forge
lcms2 2.11 hbd6801e_0 conda-forge
ld_impl_linux-64 2.35 h769bd43_9 conda-forge
libblas 3.8.0 17_openblas conda-forge
libcblas 3.8.0 17_openblas conda-forge
libclang 10.0.1 default_hde54327_1 conda-forge
libcurl 7.71.1 hcdd3856_6 conda-forge
libdap4 3.20.6 h1d1bd15_1 conda-forge
libedit 3.1.20191231 h14c3975_1.conda
libev 4.33 h516909a_1 conda-forge
libevent 2.1.10 hcdb4288_2 conda-forge
libffi 3.2.1 he1b5a44_1007 conda-forge
libgcc-ng 9.1.0 hdf63c60_0.conda
libgdal 3.1.2 hb2a6f5f_1 conda-forge
libgfortran-ng 7.5.0 hdf63c60_16 conda-forge
libiconv 1.16 h516909a_0 conda-forge
libkml 1.3.0 h74f7ee3_1012 conda-forge
liblapack 3.8.0 17_openblas conda-forge
libllvm10 10.0.1 he513fc3_3 conda-forge
libnetcdf 4.7.4 nompi_h84807e1_105 conda-forge
libnghttp2 1.41.0 h8cfc5f6_2 conda-forge
libopenblas 0.3.10 pthreads_hb3c22a3_4 conda-forge
libpng 1.6.37 hed695b0_2 conda-forge
libpq 12.3 h5513abc_0 conda-forge
libsodium 1.0.18 h516909a_0 conda-forge
libspatialindex 1.9.3 he1b5a44_3 conda-forge
libspatialite 4.3.0a h57f1b35_1039 conda-forge
libssh2 1.9.0 hab1572f_5 conda-forge
libstdcxx-ng 9.1.0 hdf63c60_0.conda
libtiff 4.1.0 hc7e4089_6 conda-forge
libuuid 2.32.1 h14c3975_1000 conda-forge
libwebp-base 1.1.0 h516909a_3 conda-forge
libxcb 1.13 h14c3975_1002 conda-forge
libxkbcommon 0.10.0 he1b5a44_0 conda-forge
libxml2 2.9.10 h68273f3_2 conda-forge
libxslt 1.1.33 h572872d_1 conda-forge
lxml 4.5.2 py38hbb43d70_0 conda-forge
lz4-c 1.9.2 he1b5a44_3 conda-forge
markupsafe 1.1.1 py38h1e0a361_1 conda-forge
matplotlib 3.3.2 0 conda-forge
matplotlib-base 3.3.2 py38h91b0d89_0 conda-forge
mistune 0.8.4 py38h1e0a361_1001 conda-forge
munch 2.5.0 py_0 conda-forge
mysql-common 8.0.21 2 conda-forge
mysql-libs 8.0.21 hf3661c5_2 conda-forge
nbclient 0.5.0 py_0 conda-forge
nbconvert 6.0.3 py38h32f6830_0 conda-forge
nbformat 5.0.7 py_0 conda-forge
ncurses 6.2 he6710b0_1.conda
nest-asyncio 1.4.0 py_0 conda-forge
notebook 6.1.4 py38h32f6830_0 conda-forge
nspr 4.28 he1b5a44_0 conda-forge
nss 3.57 he751ad9_0 conda-forge
numpy 1.19.1 py38hbc27379_2 conda-forge
olefile 0.46 py_0 conda-forge
openjpeg 2.3.1 h981e76c_3 conda-forge
openssl 1.1.1g h516909a_1 conda-forge
packaging 20.4 pyh9f0ad1d_0 conda-forge
pandas 1.1.2 py38h950e882_0 conda-forge
pandoc 2.10.1 h516909a_0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
parso 0.5.2 py_0.conda
pcre 8.44 he1b5a44_0 conda-forge
pexpect 4.8.0 py38h32f6830_1 conda-forge
pickleshare 0.7.5 py38h32f6830_1001 conda-forge
pillow 7.2.0 py38h9776b28_1 conda-forge
pip 20.2.2 py38_0.conda
pixman 0.38.0 h516909a_1003 conda-forge
poppler 0.89.0 h4190859_1 conda-forge
poppler-data 0.4.9 1 conda-forge
postgresql 12.3 h8573dbc_0 conda-forge
proj 7.1.0 h966b41f_1 conda-forge
prometheus_client 0.8.0 pyh9f0ad1d_0 conda-forge
prompt-toolkit 3.0.7 py_0 conda-forge
prompt_toolkit 3.0.7 0 conda-forge
pthread-stubs 0.4 h14c3975_1001 conda-forge
ptyprocess 0.6.0 py_1001 conda-forge
pycparser 2.20 pyh9f0ad1d_2 conda-forge
pygments 2.7.1 py_0 conda-forge
pyopenssl 19.1.0 py_1 conda-forge
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pyproj 3.5.0 <pip>
pyproj 2.6.1.post1 py38h8e47818_1 conda-forge
pyqt 5.12.3 py38ha8c2ead_3 conda-forge
pyrsistent 0.17.3 py38h1e0a361_0 conda-forge
pysocks 1.7.1 py38h32f6830_1 conda-forge
python 3.8.5 h1103e12_8_cpython conda-forge
python-dateutil 2.8.1 py_0 conda-forge
python_abi 3.8 1_cp38 conda-forge
pytz 2020.1 pyh9f0ad1d_0 conda-forge
PyYAML 6.0.1 <pip>
pyzmq 19.0.2 py38ha71036d_0 conda-forge
qt 5.12.9 h1f2b2cb_0 conda-forge
qtconsole 4.7.7 pyh9f0ad1d_0 conda-forge
qtpy 1.9.0 py_0 conda-forge
readline 8.0 h7b6447c_0.conda
requests 2.24.0 pyh9f0ad1d_0 conda-forge
rtree 0.9.4 py38h08f867b_1 conda-forge
send2trash 1.5.0 py_0 conda-forge
sentinelRequest 0.0.4.dev16+g2562afb <pip>
setuptools 49.6.0 py38_0.conda
shapely 1.7.1 py38hc7361b7_0 conda-forge
six 1.15.0 pyh9f0ad1d_0 conda-forge
sqlite 3.33.0 h62c20be_0.conda
tbb 2020.2 hc9558a2_0 conda-forge
terminado 0.8.3 py38h32f6830_1 conda-forge
testpath 0.4.4 py_0 conda-forge
tiledb 2.0.8 h3effe38_0 conda-forge
tk 8.6.10 hbc83047_0.conda
tornado 6.0.4 py38h1e0a361_1 conda-forge
tqdm 4.49.0 <pip>
traitlets 5.0.4 py_0 conda-forge
tzcode 2020a h516909a_0 conda-forge
urllib3 1.25.10 py_0 conda-forge
wcwidth 0.2.5 pyh9f0ad1d_1 conda-forge
webencodings 0.5.1 py_1 conda-forge
wheel 0.35.1 py_0.conda
widgetsnbextension 3.5.1 py38h32f6830_1 conda-forge
xerces-c 3.2.3 hfe33f54_1 conda-forge
xorg-kbproto 1.0.7 h14c3975_1002 conda-forge
xorg-libice 1.0.10 h516909a_0 conda-forge
xorg-libsm 1.2.3 h84519dc_1000 conda-forge
xorg-libx11 1.6.12 h516909a_0 conda-forge
xorg-libxau 1.0.9 h14c3975_0 conda-forge
xorg-libxdmcp 1.1.3 h516909a_0 conda-forge
xorg-libxext 1.3.4 h516909a_0 conda-forge
xorg-libxrender 0.9.10 h516909a_1002 conda-forge
xorg-renderproto 0.11.1 h14c3975_1002 conda-forge
xorg-xextproto 7.3.0 h14c3975_1002 conda-forge
xorg-xproto 7.0.31 h14c3975_1007 conda-forge
xz 5.2.5 h7b6447c_0.conda
zeromq 4.3.2 he1b5a44_3 conda-forge
zipp 3.1.0 py_0 conda-forge
zlib 1.2.11 h7b6447c_3.conda
zstd 1.4.5 h6597ccf_2 conda-forge
Screenshots
📈 Expected behavior
📎 Additional context
add notebook of example
notebook to explain how to use the client.
- associated sphinx documentation.
query with different overlapping areas will remove duplicate results -> under estimation per area
behavior observed 10 June with latest version :
query with different overlapping areas will remove duplicate results.
If one SAFE belongs to the first area, it will not be shown for the second area.
behavior expected:
keep track of all results dropping duplicates should take into account the query identification number.
query CDSE with different values in `name` column of the GeoDataFrame return None
it is an issue when one have a listing of SAFENAME and want to download them using the CDSE hash id.
add warning if we reach 1000 product in a query
conda release
Indicate which parameters are optional for the fetch_data function
It is not quite clear which columns are necessary in the given DataFrame when using fetch_data.
Which columns must be provided but can be set to None
Which columns can be omitted
It would be great to document that in the fetch_data doctstring.
Provide a regex in "name"
Currently, we can provide a string in the column "name", to match results. All results that contains the string given in "name" will be returned.
It would be nice to be able to provide a regex instead, to allow more flexible matching.
test multithread downloads
PyPi release
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.