ubc-moad/moad_tools

A Python package of shared code developed by the UBC EOAS MOAD group.

UBC EOAS MOAD Group Tools Package

Continuous Integration
Documentation
Package
Meta

The UBC EOAS MOAD Group Tools package (moad_tools) is a collection of Python modules that facilitate code reuse for the UBC EOAS MOAD Group.

Documentation for the package is in the docs/ directory and is rendered at http://ubc-moad-tools.readthedocs.org/en/latest/.

License

The UBC EOAS MOAD Group Tools package and documentation are copyright 2018 – present by the UBC EOAS MOAD Group and The University of British Columbia.

They are licensed under the Apache License, Version 2.0. https://www.apache.org/licenses/LICENSE-2.0 Please see the LICENSE file for details of the license.

moad_tools's Issues

Random-oil-spills RuntimeWarning

Fiona 1.8.18 has introduced the following RutimeWarning for every iteration in random-oil-spills:

  for feature in features_lst:
/home/rmueller/miniconda3/envs/moad-tools/lib/python3.9/site-packages/geopandas/geodataframe.py:577: RuntimeWarning: Sequential read of iterator was interrupted. Resetting iterator. This can negatively impact the performance.

According to this issue in Fiona, this RuntimeWarning is not a problem, per se, and can be ignored. There is a Fiona 1.8.19, and I’m testing that option now, though it’s not looking promising:

smelt:envs$ conda env update -f ./environment-dev.yaml 
Collecting package metadata (repodata.json): done
Solving environment: | 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
Examining conflict for xorg-libice cairo poppler xorg-libsm: : 203it [15:21,  2.Examining conflict for cairo poppler xorg-libxrender: : 204it [15:21,  2.06it/s]Examining conflict for cairo xorg-renderproto xorg-libxrender: : 205it [15:21,  Examining conflict for arrow black pytest typing_extensions: : 206it [15:21,  2.Examining conflict for arrow matplotlib: : 207it [15:21,  2.06it/s]          failed                                                                     
Solving environment: \ 
Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
Examining conflict for shapely py scipy more-itertools packaging urllib3 sphi
...

One “solution” is to force Fiona 1.8.17. I’ve tested this and it runs smoothly without the series of RuntimeWarnings. I haven’t yet compared the spill .csv using 1.8.17 with the one using 1.8.18, though I can’t imagine there would be a difference given that it’s just a RuntimeWarning.

When I update environment-dev.yaml to include Fiona 1.8.17, I get:

Downloading and Extracting Packages
scipy-1.6.2          | 20.7 MB   | ##################################### | 100%
sphinx_rtd_theme-0.5 | 6.3 MB    | ##################################### | 100%
gdal-3.1.4           | 1.5 MB    | ##################################### | 100%
fiona-1.8.17         | 1.0 MB    | ##################################### | 100%
libgdal-3.1.4        | 12.6 MB   | ##################################### | 100%
pandas-1.2.4         | 12.1 MB   | ##################################### | 100%
sphinx-3.5.4         | 1.4 MB    | ##################################### | 100%
netcdf4-1.5.6        | 560 KB    | ##################################### | 100%
pytest-6.2.3         | 434 KB    | ##################################### | 100%
rasterio-1.2.1       | 8.3 MB    | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Installing pip dependencies: | Ran pip subprocess with arguments:
...

(moad-tools) smelt:moad_tools$ conda list
# packages in environment at /home/rmueller/miniconda3/envs/moad-tools:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       1_gnu    conda-forge
affine                    2.3.0                      py_0    conda-forge
alabaster                 0.7.12                     py_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
arrow                     1.0.3            py39hf3d152e_0    conda-forge
attrs                     20.3.0             pyhd3deb0d_0    conda-forge
babel                     2.9.0              pyhd3deb0d_0    conda-forge
black                     20.8b1                     py_1    conda-forge
blosc                     1.21.0               h9c3ff4c_0    conda-forge
boost-cpp                 1.74.0               hc6e9bd1_2    conda-forge
brotlipy                  0.7.0           py39h3811e60_1001    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.17.1               h7f98852_1    conda-forge
ca-certificates           2020.12.5            ha878542_0    conda-forge
cairo                     1.16.0            h6cf1ce9_1008    conda-forge
certifi                   2020.12.5        py39hf3d152e_1    conda-forge
cffi                      1.14.5           py39he32792d_0    conda-forge
cfitsio                   3.470                hb418390_7    conda-forge
cftime                    1.4.1            py39hce5d2b2_0    conda-forge
chardet                   4.0.0            py39hf3d152e_1    conda-forge
click                     7.1.2              pyh9f0ad1d_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.1              pyhd8ed1ab_0    conda-forge
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
coverage                  5.5              py39h3811e60_0    conda-forge
cryptography              3.4.6            py39hbca0aa6_0    conda-forge
curl                      7.75.0               h979ede3_0    conda-forge
cycler                    0.10.0                     py_2    conda-forge
dataclasses               0.8                pyhc8e2a94_1    conda-forge
dbus                      1.13.6               hfdff14a_1    conda-forge
docutils                  0.16             py39hf3d152e_3    conda-forge
expat                     2.2.10               h9c3ff4c_0    conda-forge
fiona                     1.8.17           py39h645c05b_1    conda-forge
fontconfig                2.13.1            hba837de_1004    conda-forge
freetype                  2.10.4               h0708190_1    conda-forge
freexl                    1.0.6                h7f98852_0    conda-forge
gdal                      3.1.4            py39h3f36f43_8    conda-forge
geopandas                 0.9.0              pyhd8ed1ab_0    conda-forge
geos                      3.9.1                h9c3ff4c_2    conda-forge
geotiff                   1.6.0                h2b14fbe_4    conda-forge
gettext                   0.19.8.1          h0b5b191_1005    conda-forge
giflib                    5.2.1                h36c2ea0_2    conda-forge
glib                      2.68.0               h9c3ff4c_2    conda-forge
glib-tools                2.68.0               h9c3ff4c_2    conda-forge
gst-plugins-base          1.18.4               h29181c9_0    conda-forge
gstreamer                 1.18.4               h76c114f_0    conda-forge
hdf4                      4.2.13            h10796ff_1004    conda-forge
hdf5                      1.10.6          nompi_h6a2412b_1114    conda-forge
icu                       68.1                 h58526e2_0    conda-forge
idna                      2.10               pyh9f0ad1d_0    conda-forge
imagesize                 1.2.0                      py_0    conda-forge
importlib-metadata        3.10.1                   pypi_0    pypi
iniconfig                 1.1.1              pyh9f0ad1d_0    conda-forge
jinja2                    2.11.3             pyh44b312d_0    conda-forge
jpeg                      9d                   h36c2ea0_0    conda-forge
json-c                    0.15                 h98cffda_0    conda-forge
kealib                    1.4.14               hcc255d8_2    conda-forge
kiwisolver                1.3.1            py39h1a9c180_1    conda-forge
krb5                      1.17.2               h926e7f8_0    conda-forge
lcms2                     2.12                 hddcbb42_0    conda-forge
ld_impl_linux-64          2.35.1               hea4e1c9_2    conda-forge
libblas                   3.9.0                8_openblas    conda-forge
libcblas                  3.9.0                8_openblas    conda-forge
libclang                  11.1.0          default_ha53f305_0    conda-forge
libcurl                   7.75.0               hc4aaa36_0    conda-forge
libdap4                   3.20.6               hd7c4107_2    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               hcdb4288_3    conda-forge
libffi                    3.3                  h58526e2_2    conda-forge
libgcc-ng                 9.3.0               h2828fa1_18    conda-forge
libgdal                   3.1.4                h38ff51b_8    conda-forge
libgfortran-ng            9.3.0               hff62375_18    conda-forge
libgfortran5              9.3.0               hff62375_18    conda-forge
libglib                   2.68.0               h3e27bee_2    conda-forge
libgomp                   9.3.0               h2828fa1_18    conda-forge
libiconv                  1.16                 h516909a_0    conda-forge
libkml                    1.3.0             h02e6976_1012    conda-forge
liblapack                 3.9.0                8_openblas    conda-forge
libllvm11                 11.1.0               hf817b99_0    conda-forge
libnetcdf                 4.7.4           nompi_h56d31a8_107    conda-forge
libnghttp2                1.43.0               h812cca2_0    conda-forge
libopenblas               0.3.12          pthreads_h4812303_1    conda-forge
libpng                    1.6.37               h21135ba_2    conda-forge
libpq                     13.1                 hfd2b0eb_2    conda-forge
librttopo                 1.1.0                h1185371_6    conda-forge
libspatialindex           1.9.3                h9c3ff4c_3    conda-forge
libspatialite             5.0.1                he52d314_3    conda-forge
libssh2                   1.9.0                ha56f1ee_6    conda-forge
libstdcxx-ng              9.3.0               h6de172a_18    conda-forge
libtiff                   4.2.0                hdc55705_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libwebp-base              1.2.0                h7f98852_2    conda-forge
libxcb                    1.13              h7f98852_1003    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.10               h72842e0_3    conda-forge
lz4-c                     1.9.3                h9c3ff4c_0    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
markupsafe                1.1.1            py39h3811e60_3    conda-forge
matplotlib                3.3.4            py39hf3d152e_0    conda-forge
matplotlib-base           3.3.4            py39h2fa2bec_0    conda-forge
moad-tools                21.1.dev0                 dev_0    <develop>
mock                      4.0.3            py39hf3d152e_1    conda-forge
more-itertools            8.7.0              pyhd8ed1ab_0    conda-forge
munch                     2.5.0                      py_0    conda-forge
mypy_extensions           0.4.3            py39hf3d152e_3    conda-forge
mysql-common              8.0.23               ha770c72_1    conda-forge
mysql-libs                8.0.23               h935591d_1    conda-forge
ncurses                   6.2                  h58526e2_4    conda-forge
netcdf4                   1.5.6           nompi_py39h36800e2_101    conda-forge
nspr                      4.30                 h9c3ff4c_0    conda-forge
nss                       3.63                 hb5efdd6_0    conda-forge
numexpr                   2.7.3            py39hde0f152_0    conda-forge
numpy                     1.20.1           py39hdbf815f_0    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openjpeg                  2.4.0                hf7af979_0    conda-forge
openssl                   1.1.1k               h7f98852_0    conda-forge
packaging                 20.9               pyh44b312d_0    conda-forge
pandas                    1.2.4            py39hde0f152_0    conda-forge
pathspec                  0.8.1              pyhd3deb0d_0    conda-forge
pcre                      8.44                 he1b5a44_0    conda-forge
pillow                    8.1.2            py39hf95b381_0    conda-forge
pip                       21.0.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pluggy                    0.13.1           py39hf3d152e_4    conda-forge
poppler                   0.89.0               h2de54a5_5    conda-forge
poppler-data              0.4.10                        0    conda-forge
postgresql                13.1                 h6303168_2    conda-forge
proj                      7.2.0                h277dcde_2    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
py                        1.10.0             pyhd3deb0d_0    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pygments                  2.8.1              pyhd8ed1ab_0    conda-forge
pyopenssl                 20.0.1             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7              pyh9f0ad1d_0    conda-forge
pyproj                    3.0.1            py39h50a64a9_0    conda-forge
pyqt                      5.12.3           py39hf3d152e_7    conda-forge
pyqt-impl                 5.12.3           py39h0fcd23e_7    conda-forge
pyqt5-sip                 4.19.18          py39he80948d_7    conda-forge
pyqtchart                 5.12             py39h0fcd23e_7    conda-forge
pyqtwebengine             5.12.1           py39h0fcd23e_7    conda-forge
pysocks                   1.7.1            py39hf3d152e_3    conda-forge
pytables                  3.6.1            py39hf6dc253_3    conda-forge
pytest                    6.2.3            py39hf3d152e_0    conda-forge
pytest-cov                2.11.1             pyh44b312d_0    conda-forge
pytest-randomly           3.7.0                    pypi_0    pypi
python                    3.9.2           hffdb5ce_0_cpython    conda-forge
python-dateutil           2.8.1                      py_0    conda-forge
python_abi                3.9                      1_cp39    conda-forge
pytz                      2021.1             pyhd8ed1ab_0    conda-forge
pyyaml                    5.4.1            py39h3811e60_0    conda-forge
qt                        5.12.9               hda022c4_4    conda-forge
rasterio                  1.2.1            py39hb73aa6c_0    conda-forge
readline                  8.0                  he28a2e2_2    conda-forge
regex                     2021.3.17        py39h3811e60_0    conda-forge
requests                  2.25.1             pyhd3deb0d_0    conda-forge
rtree                     0.9.7            py39hb102c33_1    conda-forge
scipy                     1.6.2            py39hee8e79c_0    conda-forge
setuptools                49.6.0           py39hf3d152e_3    conda-forge
shapely                   1.7.1            py39ha61afbd_4    conda-forge
six                       1.15.0             pyh9f0ad1d_0    conda-forge
snowballstemmer           2.1.0              pyhd8ed1ab_0    conda-forge
snuggs                    1.4.7                      py_0    conda-forge
sphinx                    3.5.4              pyh44b312d_0    conda-forge
sphinx_rtd_theme          0.5.2              pyhd8ed1ab_1    conda-forge
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge
sqlite                    3.35.2               h74cdb3f_0    conda-forge
tiledb                    2.2.7                h91fcb0e_0    conda-forge
tk                        8.6.10               h21135ba_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tornado                   6.1              py39h3811e60_1    conda-forge
typed-ast                 1.4.2            py39h3811e60_0    conda-forge
typing_extensions         3.7.4.3                    py_0    conda-forge
tzcode                    2021a                h7f98852_1    conda-forge
tzdata                    2021a                he74cb21_0    conda-forge
urllib3                   1.26.4             pyhd8ed1ab_0    conda-forge
wheel                     0.36.2             pyhd3deb0d_0    conda-forge
xarray                    0.17.0             pyhd8ed1ab_0    conda-forge
xerces-c                  3.2.3                h9d8b166_2    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.7.0                h7f98852_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h7f98852_1    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h7f98852_1002    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xz                        5.2.5                h516909a_1    conda-forge
yaml                      0.2.5                h516909a_0    conda-forge
zipp                      3.4.1                    pypi_0    pypi
zlib                      1.2.11            h516909a_1010    conda-forge
zstd                      1.4.9                ha95c52a_0    conda-forge

(note: this last list was printed while the conda update using Fiona 1.8.18, mentioned above, was in process, which I don’t think would change anything above--given that it hasn’t finished--but I mention anyway just in case).

The question is (assuming Fiona 1.8.19 is a no-go): Do we want to run with Fiona 1.8.17 or tolerate the RuntimeWarnings with Fiona 1.8.18?

AttributeError: module 'numpy' has no attribute 'int'

Message in testing log:

np.int was a deprecated alias for the builtin int. To avoid this error in existing code, use int by itself. Doing this will not modify any behavior and is safe. When replacing np.int, you may wish to use e.g. np.int64 or np.int32 to specify the precision. If you wish to review your current use, check the release note link for additional information.
The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations.

Review use of reStructuredText :kbd: role in docs

sphinx-rtd-theme has change styling of :kdb: items to look like a keycap. That is not appropriate for things like machine names; e.g. :kbd:'salish'.