creare-com/pydem

Python library for Global Hydrology Analysis. Used to calculate upstream contributing area, aspect, slope, and topographic wetness index.

pydem's Issues

Error in calc_uca()

When I try to run calc_uca, I get the following error:

pydem/cyfuncs/cyutils.pyx in pydem.cyfuncs.cyutils.drain_area (/private/var/folders/1j/r_xd9vp15td_mdt_3rgrx13w0000gp/T/pip-rmLTHG-build/pydem/cyfuncs/cyutils.cpp:2470)() ValueError: Item size of buffer (8 bytes) does not match size of 'int' (4 bytes)

Installed pydem via pip install --ignore-installed git+git://github.com/creare-com/pydem.git@develop as in here https://github.com/creare-com/pydem/issues/7 to solve the same problem: TypeError: Cannot cast ufunc add output from dtype('float64') to dtype('bool') with casting rule 'same_kind'.

Python 2.7.12, MacOS 10.12.3, 64-bit.

Note: GeoTIFFs I provide as an input are of float32 type, but calc_slopes_directions outputs float64.

cl.exe failed with exit status 2

I am attempting to install pyDEM via pip.

I have Python 2.7.9. I have also installed the necessary numpy, scipy, and GDAL packages.

When I try to install it, I get the following error. Anyone have know how to make this work?

traits/ctraits.c(956) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(2221) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(2254) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3167) : warning C4244: 'initializing' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3189) : warning C4244: 'initializing' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3513) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3597) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3808) : warning C4244: 'initializing' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3814) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3824) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(3927) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(4131) : warning C4244: '=' : conversion from 'Py_ssize_t' to 'int', possible loss of data

traits/ctraits.c(5477) : error C2079: 'ctraitsmodule' uses undefined struct 'PyModuleDef'

traits/ctraits.c(5478) : error C2065: 'PyModuleDef_HEAD_INIT' : undeclared identifier

traits/ctraits.c(5478) : error C2099: initializer is not a constant

traits/ctraits.c(5478) : warning C4047: 'initializing' : 'int' differs in levels of indirection from 'char [8]'

traits/ctraits.c(5478) : error C2078: too many initializers

traits/ctraits.c(5488) : warning C4013: 'PyModule_Create' undefined; assuming extern returning int

traits/ctraits.c(5488) : warning C4047: '=' : 'PyObject *' differs in levels of indirection from 'int'

traits/ctraits.c(5490) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5497) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5503) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5511) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5516) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5537) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5542) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5547) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5554) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5559) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5564) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

traits/ctraits.c(5568) : warning C4098: 'PyInit_ctraits' : 'void' function returning a value

error: command 'C:\\Users\\User\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2

How to deal with no place to drain warning

While calculating UCA, I'm getting User warning, "Warning 2416 pits had no place to drain to in this chunk
"chunk" % len(warn_pits))". How to handle this kind of warning. I'm getting this error on many chunks. Any suggestion?

KeyError upon Instantiating an instance of the DEMProcessor class --Projection

I am running PyDEM (0.2.0) on Python (2.7.14) and trying to follow the instructions on the readme.md file. Below I have listed the commands I used, the output, and the input elevation file. I am getting the KeyError with the third command. It looks like it has something to do with the projection files. Has anyone else had this issue and if so, how did you solve it. I tried to search this error with Google but did not find anything useful.

Here are the commands I typed in the Python Console:

from pydem.dem_processing import DEMProcessor
boy = 'elevationUTM.tif'
dem_proc = DEMProcessor(boy)

Then I receive the messages including the key error at the end:

Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\Python27\lib\site-packages\pydem\dem_processing.py", line 565, in __init__
    elev, = elev_file.raster_layers
  File "C:\Python27\lib\site-packages\traits\has_traits.py", line 895, in decorator
    self.__dict__[ name ] = result = function( self )
  File "C:\Python27\lib\site-packages\pydem\reader\gdal_reader.py", line 195, in _get_raster_layers
    for raster_band in raster_bands]
  File "C:\Python27\lib\site-packages\pydem\reader\gdal_reader.py", line 143, in _raster_layer_from_raster_band
    layer.grid_coordinates = self.grid_coordinates
  File "C:\Python27\lib\site-packages\traits\has_traits.py", line 895, in decorator
    self.__dict__[ name ] = result = function( self )
  File "C:\Python27\lib\site-packages\pydem\reader\gdal_reader.py", line 130, in _get_grid_coordinates
    wkt = d_wkt_to_name[wkt_]
KeyError: 'PROJCS["WGS_1984_UTM_Zone_16N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-87],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AUTHORITY["EPSG","32616"]]'

Here is the input elevation model I used (this file was created with ArcGISPro using the project raster command and the projection was WGS_1984_UTM_Zone_16N):
elevationUTM.tif.zip

I have tried the solution recommended by mpu-creare but this did not work. Here is the link for that GitHub Issue.
https://github.com/creare-com/pydem/issues/1

Lowering error using pyDEM dem processor

I'm in Win7 64, Anaconda Python installation, pip installed PyDEM.

I'm getting an error when calling .calc_slopes_directions() on a DEMProcessor instance (see below).

I'm aware of this question; that seems not to be the same problem I have.

This LoweringError: slc0.8 seems to be related to Numba.

This is the code I enter and the error I get:

from pydem.dem_processing import DEMProcessor

os.chdir(r'D:\my_path')

a_tile = r'31an2_clipped_wgs84.tif'

dem_proc = DEMProcessor(a_tile)

mag, aspect = dem_proc.calc_slopes_directions()
starting slope/direction calculation for chunk 1 [0:516, 0:516]
Traceback (most recent call last):

  File "<ipython-input-13-6dc0db5771d2>", line 1, in <module>
    mag, aspect = dem_proc.calc_slopes_directions()

  File "C:\Program Files\Anaconda2\lib\site-packages\pydem\dem_processing.py", line 853, in calc_slopes_directions
    self.dY[te:be-1])

  File "C:\Program Files\Anaconda2\lib\site-packages\pydem\dem_processing.py", line 879, in _slopes_directions
    return self._tarboton_slopes_directions(data, dX, dY)

  File "C:\Program Files\Anaconda2\lib\site-packages\pydem\dem_processing.py", line 890, in _tarboton_slopes_directions
    self.facets, self.ang_adj)

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\dispatcher.py", line 285, in _compile_for_args
    return self.compile(tuple(argtypes))

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\dispatcher.py", line 531, in compile
    cres = self._compiler.compile(args, return_type)

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\dispatcher.py", line 80, in compile
    flags=flags, locals=self.locals)

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\compiler.py", line 725, in compile_extra
    return pipeline.compile_extra(func)

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\compiler.py", line 369, in compile_extra
    return self.compile_bytecode(bc, func_attr=self.func_attr)

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\compiler.py", line 378, in compile_bytecode
    return self._compile_bytecode()

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\compiler.py", line 690, in _compile_bytecode
    return self._compile_core()

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\compiler.py", line 677, in _compile_core
    res = pm.run(self.status)

  File "C:\Program Files\Anaconda2\lib\site-packages\numba\compiler.py", line 257, in run
    raise patched_exception

LoweringError: slc0.8

How to install taudem?

I tried installing taudem through the Windows installer downloaded from original website. Even after installing it, when I run an example code it raises the following exception:
File "C:\Anaconda3\envs\codefundo\lib\site-packages\pydem\taudem\taudem.py", line 33, in _run
raise Exception("Taudem did not complete. Is it installed properly?")
Exception: Taudem did not complete. Is it installed properly?

Installation error: Microsoft Visual C++ 14.0 is required.

Hi all. I get the following errors when trying to install PyDEM from pip on Windows, with the Anaconda distribution of Python:

Running setup.py bdist_wheel for pyDEM: finished with status 'error'
[...]
building 'pydem.cyfuncs.cyutils' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools

...which I can't install on this corporate computer. I assume I'll get the same issue trying to build from source too.

Might I suggest you look into the following for an eventual release? Would be interesting to:

1. Upload pre-built wheels to pypi, and/or
2. Create a conda-forge recipe for the anaconda distribution.

Cheers!

KeyError reading coordinate strings

I cant read a dem file.

I have installed pydem using pip and all the requirements (numba, etc) via conda.

When I try

dem_proc = DEMProcessor(filename_to_elevation_geotiff)

I am receiving a key error with an etopo dem:

/home/emiliano/anaconda/lib/python2.7/site-packages/pydem/reader/gdal_reader.pyc
in get_grid_coordinates(self)
128
129 wkt = dataset.GetProjection()
--> 130 wkt = d_wkt_to_name[wkt_]
131
132

KeyError: 'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS
84",6378137,298.2572235604902,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433],AUTHORITY["EPSG","4326"]]'

And, similar error message for aster dem:

KeyError: 'GEOGCS["WGS
84",DATUM["unknown",SPHEROID["WGS84",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]]'

I have also tried converting .bil to .tiff using gdal_convert, and I get
the same keyerror message.

Check for python version before installing

Pydem requires python 2.7 but somehow I was able to install and import the library using python 3.5. Needless to say my application failed.

Unrelated feature request: It would be useful to be able to instantiate DEMProcessor from a 2D numpy array rather than passing the name to a tif file.

How can we use UCA calculations to compute Runoff Volume?

I have calculated runoff based on precipitation data using SCS Curve Number Method. Now I want to compute the volume of runoff and drainage capacity. Any suggestion?

Trouble saving outputs to file

Hello,

I am a graduate researcher at the University of Virginia and I am working on designing/optimizing a terrain analysis model used to identify wetlands. I am just beginning to move this model out of ArcGIS and I think the pyDEM library is a great tool to calculate the TWI and its derivatives for my study sites. I'm starting off just working with the test_NN064_013_elev.tif file before processing the elevation tiles for my study sites. Using the test tif, I try to create a DEMProcessor object, calculate the slope and directions, plot the slope array, and save the slope array to a file. My code is below:

filename_to_elevation_geotiff = 'test_NN064_013_elev.tif'

dem_proc = DEMProcessor(filename_to_elevation_geotiff)

mag, aspect = dem_proc.calc_slopes_directions()

print dem_proc.mag

plt.imshow(dem_proc.direction)

if not os.path.exists('mag'):
os.mkdir('mag')
dem_proc.save_slope('.')
print 'done'

---

When I run this code, I get the following output:

starting slope/direction calculation
[[ 1.57748599e-05 1.57748599e-05 1.56843030e-05 ..., 1.56843030e-05
1.57748599e-05 1.57748599e-05]
[ 1.57748599e-05 1.57748599e-05 1.56843030e-05 ..., 1.56843030e-05
1.57748599e-05 1.57748599e-05]
[ 1.58631500e-05 1.58631500e-05 1.57733022e-05 ..., 1.57733022e-05
1.58631500e-05 1.58631500e-05]
...,
[ 1.56804838e-05 1.56804838e-05 1.55954034e-05 ..., 1.55954034e-05
1.56804838e-05 1.56804838e-05]
[ 1.55911111e-05 1.55911111e-05 1.55055366e-05 ..., 1.55055366e-05
1.55911111e-05 1.55911111e-05]
[ 1.55911111e-05 1.55911111e-05 1.55055366e-05 ..., 1.55055366e-05
1.55911111e-05 1.55911111e-05]]
<<<<<<<< gdalwarp -multi -wm 2000 -co BIGTIFF=YES -of GTiff -co compress=lzw -co TILED=YES -wo OPTIMIZE_SIZE=YES -r near -t_srs EPSG:4326 .\mag\N45W-73_N46W-72_mag_tmp.tif .\mag\N45W-73_N46W-72_mag.tif >>>>>>>>
done

A plot is created (attached) and a "mag" folder is created in my directory. However, this folder remains empty and I cannot find the newly created mag geotiff anywhere on my computer.

We think that this line :

https://github.com/creare-com/pydem/blob/master/pydem/dem_processing.py#L682

which removes the tmp file may be deleting the output of the save_slope method. We have tried to edit the dem_processing.py file to comment out line 682, but the edits (or other test edits) do not seem to affect my code, and I get the same result.

What do you think is causing this?

Thanks,
Gina

Python 3 release open issues

• Need to fix Geotiff write out when the tiles are not contiguous
• Need mechanism to pass parameters through compute manager to demprocessor
• Need a helper function to compute everything needed to run any of the process functions
• Need to fix the order of operations for the edge-fixing so that each tile is at least visited once before any other tile is processed a second time. That just gets 99% of the tile-to-tile errors

Working with Nan values

I tried pyDEM on SRTM data, I'm able to calculate TWI, but in the final output, I'm getting Nan values as well. Can you please help me how to resolve this issue or is there any pre-processing required? Also can you please add a comment on visualization part of the TWI on the map, like Folium etc?

Type casting problems with SRTM DEM

I was hoping to compute slope and aspect for some digital elevation models from the NASA SRTM mission, and ran into a type error that seems to be related to the _find_flats_edges function.

Here's a reproducible example Jupyter notebook that demonstrates the problem: https://gist.github.com/mbjoseph/9934454330b5715db9529502cf415f72#file-pydem-srtm-example-ipynb

A TypeError is raised:

TypeError: Cannot cast ufunc add output from dtype('float64') to dtype('bool') with casting rule 'same_kind'

It seems like similar issues have arisen elsewhere, e.g., numpy/numpy#7225

Cannot open source file: 'pydem\cyfuncs\cyutils.c'

Hi there,

I am trying to install pydem on a Windows 7 system, and I am getting the error "Cannot open source file: 'pydem\cyfuncs\cyutils.c'" Is that something that should be copied from my cython installation, or should that come with the pydem download?

Thanks,
Stephanie

Runtime Error

I'm able to install package and import functions. I instantiated the instance with the raster file but getting maximum recursion depth error while calculating aspect and slope magnitude. I'm using ASTER-DEM data. Can anyone help me to resolve this one?

Problems with Taudem

Hi, the link in your instructions for the Taudem source for windows: http://svn.mapwindow.org/svnroot/MapWindow4Dev/Bin/Taudem5Exe/ is not working. Trying to go around the problem, I installed Taudem following the instructions in their website for windows, and although Taudem was properly working, when I try to run pydem in one of your examples, i.e. elev, ang, test_data = get_test_data(testnum, NN) it returns the message: "Taudem did not complete. Is it installed properly?". After that I manually copy the executables from the Taudem installation found in the folder C:\Program Files\TauDEM\TauDEM5Exe to a folder you suggested: <ANACONDA_ROOT><lib>\pydem\taudem\taudem_Windows and I still get the message: "Taudem did not complete. Is it installed properly?".

Thanks for your help,