r-barnes/barnes2019-depressionhierarchy

Fast handling of depressions in flow graphs

barnes2019-depressionhierarchy's Issues

Test: code aborted after successfully producing output

I installed the prereqs and compiled the software no problem with the README information -- excellent!

I then created a test case with a DEM of my favorite place as a kid:

./build/dephier.exe AWtest/BCwest.tif AWtest/testOut AWtest/testDH 0

It ran well -- all outputs look fine -- but ended with an abort:

./build/dephier.exe AWtest/BCwest.tif AWtest/testOut AWtest/testDH 0
m Processing    = AWtest/BCwest.tif
m Output prefix = AWtest/testOut
m Output graph  = AWtest/testDH
m Ocean level   = 0
m Data width  = 130
m Data height = 170
m Data cells  = 22100
##########Getting depression hierarchy
p Adding ocean cells to priority-queue...
p Finding pit cells...
t Pit cells found in = 0.0 s
p Searching for outlets...
t Outlets found in = 0.0 s
p Constructing hierarchy from outlets...
t Time to construct Depression Hierarchy = 0.1 s
p Calculating depression marginal volumes...
p Calculating depression total volumes...
t Time to calculate volumes = 0.0 s
t Total time in depression hierarchy calculations = 0.1 s
p Moving surface water downstream...
t FlowInDepressionHierarchy: Surface water = 0.0 s
t FlowInDepressionHierarchy: Overflow time = 0.0
p Finding filled...
t FlowInDepressionHierarchy: Fill time = 0.0 s
t FlowInDepressionHierarchy = 0.0 s
Finished
IO time   = 0.13301
3 s
double free or corruption (out)
Aborted (core dumped)

Files: AWtest.zip

Add information on to DH data structure

Add (a) function(s) to post-hoc update the data structure:

• Depression area
• Depression depth (ocean = 0, everything else +1 per layer of nesting)

Install instructions

On a Mac at least, your recursive git call needs to be

git submodule update --recursive

not

git submodule --update --recursive

as written. I think.

Retain projection information

Suggested future improvement (I can volunteer myself to help, too): retaining projection information could be very helpful when building outputs from DH. This should be possible if exporting as NetCDF or GeoTiff.

Compile on Windows

Hi,

Thanks for this work!

I try to compile this program on Windows, but the compilation failed.
Done it with the following code from the Linux tutorial :
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DUSE_GDAL=On .. -G"Unix Makefiles"
make -j 8

I try several versions of GCC, but it is the same error.

When i make it, here is tje error code :
[ 10%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/src/richdem.cpp.obj
[ 20%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/src/random.cpp.obj
[ 30%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/src/gdal.cpp.obj
In file included from c:\mingw64\include\c++\12.2.0\stdlib.h:36,
from C:/Users/alexa/miniconda3/Library/include/cpl_port.h:138,
from C:/Users/alexa/miniconda3/Library/include/gdal.h:45,
from C:/Users/alexa/miniconda3/Library/include/gdal_priv.h:60,
from C:/Users/alexa/Documents/GitHub/Barnes2019-DepressionHierarchy/submodules/richdem/include/richdem/common/gdal.hpp:5,
from C:/Users/alexa/Documents/GitHub/Barnes2019-DepressionHierarchy/submodules/richdem/src/gdal.cpp:1:
c:\mingw64\include\c++\12.2.0\cstdlib:137:11: error: 'at_quick_exit' has not been declared in '::'
137 | using ::at_quick_exit;
| ^~~~~~~~~~~~~
c:\mingw64\include\c++\12.2.0\cstdlib:160:11: error: 'quick_exit' has not been declared in '::'
160 | using ::quick_exit;
| ^~~~~~~~~~
c:\mingw64\include\c++\12.2.0\stdlib.h:43:14: error: 'at_quick_exit' has not been declared in 'std'
43 | using std::at_quick_exit;
| ^~~~~~~~~~~~~
c:\mingw64\include\c++\12.2.0\stdlib.h:46:14: error: 'quick_exit' has not been declared in 'std'
46 | using std::quick_exit;
| ^~~~~~~~~~
make[2]: *** [submodules/richdem/CMakeFiles/richdem.dir/src/gdal.cpp.obj] Erreur 1
make[1]: *** [submodules/richdem/CMakeFiles/richdem.dir/all] Erreur 2
make: *** [all] Erreur 2

CMakeError.log:
CMakeError.log

Thanks for your help

Command make -j 4

Hi,

The following command "make -j 4" doesn't work when
-DUSE_GDAL=Off or if gdal library is not installed

Test output: duplicate, documentation, and suggestion

This should be a simple one. In my test run (#7: AWtest.zip), I labeled my output testOut. It returned:

• testOut-diff.tif
• testOut-filled.nc
• testOut-flooded.nc
• testOut-label.tif

These four returned files are pretty intuitive but not defined in the README.

The second and third items are identical, so these should be combined.

I am not sure of the value of returning the diff. It seems like it could be helpful -- or it could be unhelpful, if working with a very large DEM whose extra export could fill HDD space.

testOut-label.tif is obviously the set of leaf-depression labels.

Considering that you have so many outputs that are all named after this main name, I think it may make sense to do the same with the DH connectivity output, thereby changing the command-line way of operating the DH but having a more coherent set of outputs. This set of connections should be significantly smaller than the rasters, and the label.tif output will not be very useful without it.

Install on Mac

Hi Richard,

Moving this over here as I suspect it might be related to r-barnes/Barnes2020-FillSpillMerge#2.

Decided to try direct installs of richdem (on pip, fine) and dephier, and for dephier this happened. The build appears to get made OK, but then I cannot actually run the software, with the prompt complaining dyld: Library not loaded: @rpath/libomp.dylib... Reason: image not found. Note that the upgrade command also appears not to have taken here. Possibly this is also happening with r-barnes/Barnes2020-FillSpillMerge#2.

Again, same setup as at the other issue: iMac Catalina, forced downgrade of the Xcode SDK to 10.9 to get cmake firing.

(base) ➜  development git clone --recurse-submodules -j8 https://github.com/r-barnes/Barnes2019-DepressionHierarchy.git
Cloning into 'Barnes2019-DepressionHierarchy'...
remote: Enumerating objects: 45, done.
remote: Counting objects: 100% (45/45), done.
remote: Compressing objects: 100% (31/31), done.
remote: Total 376 (delta 16), reused 35 (delta 8), pack-reused 331
Receiving objects: 100% (376/376), 232.34 KiB | 975.00 KiB/s, done.
Resolving deltas: 100% (202/202), done.
Submodule 'submodules/richdem' (https://github.com/r-barnes/richdem.git) registered for path 'submodules/richdem'
Cloning into '/Users/danhobley/development/Barnes2019-DepressionHierarchy/submodules/richdem'...
remote: Enumerating objects: 27, done.        
remote: Counting objects: 100% (27/27), done.        
remote: Compressing objects: 100% (16/16), done.        
remote: Total 10977 (delta 16), reused 21 (delta 11), pack-reused 10950        
Receiving objects: 100% (10977/10977), 11.08 MiB | 2.01 MiB/s, done.
Resolving deltas: 100% (7277/7277), done.
Submodule path 'submodules/richdem': checked out '349fc363b622e68a652c44211d20fe50094ea43c'
Submodule 'docs/richdem-docs' (https://github.com/r-barnes/richdem-docs.git) registered for path 'submodules/richdem/docs/richdem-docs'
Cloning into '/Users/danhobley/development/Barnes2019-DepressionHierarchy/submodules/richdem/docs/richdem-docs'...
remote: Enumerating objects: 667, done.        
remote: Total 667 (delta 0), reused 0 (delta 0), pack-reused 667        
Receiving objects: 100% (667/667), 87.61 MiB | 2.41 MiB/s, done.
Resolving deltas: 100% (324/324), done.
Submodule path 'submodules/richdem/docs/richdem-docs': checked out '2ed8749b771924932cc62a7ca90ff06e7009f0f6'

(base) ➜  development cd Barnes2019-DepressionHierarchy 
(base) ➜  Barnes2019-DepressionHierarchy git:(master) git submodule --update --recursive
usage: git submodule [--quiet] add [-b <branch>] [-f|--force] [--name <name>] [--reference <repository>] [--] <repository> [<path>]
   or: git submodule [--quiet] status [--cached] [--recursive] [--] [<path>...]
   or: git submodule [--quiet] init [--] [<path>...]
   or: git submodule [--quiet] deinit [-f|--force] (--all| [--] <path>...)
   or: git submodule [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference <repository>] [--recursive] [--] [<path>...]
   or: git submodule [--quiet] summary [--cached|--files] [--summary-limit <n>] [commit] [--] [<path>...]
   or: git submodule [--quiet] foreach [--recursive] <command>
   or: git submodule [--quiet] sync [--recursive] [--] [<path>...]
   or: git submodule [--quiet] absorbgitdirs [--] [<path>...]

(base) ➜  Barnes2019-DepressionHierarchy git:(master) mkdir build
(base) ➜  Barnes2019-DepressionHierarchy git:(master) cd build
(base) ➜  build git:(master) cmake -DCMAKE_BUILD_TYPE=Release ..
-- The C compiler identification is Clang 4.0.1
-- The CXX compiler identification is Clang 4.0.1
-- Check for working C compiler: /Users/danhobley/opt/anaconda3/bin/x86_64-apple-darwin13.4.0-clang
-- Check for working C compiler: /Users/danhobley/opt/anaconda3/bin/x86_64-apple-darwin13.4.0-clang - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Users/danhobley/opt/anaconda3/bin/x86_64-apple-darwin13.4.0-clang++
-- Check for working CXX compiler: /Users/danhobley/opt/anaconda3/bin/x86_64-apple-darwin13.4.0-clang++ - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found NetCDF: /usr/local/lib/libnetcdf.dylib  
-- Found GDAL: /usr/local/Cellar/gdal/2.4.4_6/lib/libgdal.dylib (found version "2.4.4") 
-- Found OpenMP_C: -fopenmp=libomp (found version "3.1") 
-- Found OpenMP_CXX: -fopenmp=libomp (found version "3.1") 
-- Found OpenMP: TRUE (found version "3.1")  
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/danhobley/development/Barnes2019-DepressionHierarchy/build

(base) ➜  build git:(master) make -j 4
Scanning dependencies of target richdem
[ 28%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/include/richdem/common/random.cpp.o
[ 28%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/include/richdem/richdem.cpp.o
[ 42%] Linking CXX static library librichdem.a
[ 42%] Built target richdem
Scanning dependencies of target dephier.exe
Scanning dependencies of target dephier_paper_tests.exe
[ 71%] Building CXX object CMakeFiles/dephier_paper_tests.exe.dir/src/dephier_paper_tests.cpp.o
[ 71%] Building CXX object CMakeFiles/dephier.exe.dir/src/main.cpp.o
[ 85%] Linking CXX executable dephier.exe
[ 85%] Built target dephier.exe
[100%] Linking CXX executable dephier_paper_tests.exe
[100%] Built target dephier_paper_tests.exe

(base) ➜  build git:(master) cd ..
(base) ➜  Barnes2019-DepressionHierarchy git:(master) cd ..
(base) ➜  development cd Barnes2019-DepressionHierarchy 
(base) ➜  Barnes2019-DepressionHierarchy git:(master) ./build/dephier.exe
dyld: Library not loaded: @rpath/libomp.dylib
  Referenced from: /Users/danhobley/development/Barnes2019-DepressionHierarchy/./build/dephier.exe
  Reason: image not found
[1]    24196 abort      ./build/dephier.exe
(base) ➜  Barnes2019-DepressionHierarchy git:(master) ./build/dephier    
zsh: no such file or directory: ./build/dephier
(base) ➜  Barnes2019-DepressionHierarchy git:(master) cd build
(base) ➜  build git:(master) ls
CMakeCache.txt          cmake_install.cmake     submodules
CMakeFiles              dephier.exe
Makefile                dephier_paper_tests.exe
(base) ➜  build git:(master) dephier
zsh: command not found: dephier
(base) ➜  build git:(master) dephier.exe
zsh: command not found: dephier.exe
(base) ➜  build git:(master) ./dephier.exe
dyld: Library not loaded: @rpath/libomp.dylib
  Referenced from: /Users/danhobley/development/Barnes2019-DepressionHierarchy/build/./dephier.exe
  Reason: image not found
[1]    24263 abort      ./dephier.exe

Kerry's tests: SL=0 issue?

• It seems that Kerry's ASCII tests are failing when sea level = 0.
• run_tests.sh will not work because of the altered output name and different structure.

What is the cause of the former, and is it worth updating the latter?

Syntax errors during compilation (Ubuntu 16.04)

I attempted following the instructions in the Readme to compile dephier.exe. However, I ran into what appear to be syntax errors during compilation of dephier.hpp. I'm on Ubuntu 16.04 and, after updating for this project, have CMake 3.17.3. C and CXX compilers are GNU 5.4.0. My best guess is that it is a compiler version issue, but without recent experience in C++ I'm not sure where to start. Googling didn't immediately lead to any clues. Error message is below:

Scanning dependencies of target richdem
[ 14%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/include/richdem/richdem.cpp.o
[ 28%] Building CXX object submodules/richdem/CMakeFiles/richdem.dir/include/richdem/common/random.cpp.o
[ 42%] Linking CXX static library librichdem.a
[ 42%] Built target richdem
Scanning dependencies of target dephier.exe
Scanning dependencies of target dephier_paper_tests.exe
[ 57%] Building CXX object CMakeFiles/dephier.exe.dir/src/main.cpp.o
[ 71%] Building CXX object CMakeFiles/dephier_paper_tests.exe.dir/src/dephier_paper_tests.cpp.o
In file included from /home/mcoving/github/Barnes2019-DepressionHierarchy/src/main.cpp:1:0:
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:24:18: error: expected ‘{’ before ‘::’ token
namespace richdem::dephier {
^
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:24:20: error: ‘dephier’ in namespace ‘::’ does not name a type
namespace richdem::dephier {
^
In file included from /home/mcoving/github/Barnes2019-DepressionHierarchy/src/main.cpp:1:0:
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:268:64: error: expected ‘}’ before end of line
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:268:64: error: expected declaration before end of line
In file included from /home/mcoving/github/Barnes2019-DepressionHierarchy/src/dephier_paper_tests.cpp:1:0:
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:24:18: error: expected ‘{’ before ‘::’ token
namespace richdem::dephier {
^
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:24:20: error: ‘dephier’ in namespace ‘::’ does not name a type
namespace richdem::dephier {
^
In file included from /home/mcoving/github/Barnes2019-DepressionHierarchy/src/dephier_paper_tests.cpp:1:0:
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:268:64: error: expected ‘}’ before end of line
/home/mcoving/github/Barnes2019-DepressionHierarchy/include/dephier/dephier.hpp:268:64: error: expected declaration before end of line
CMakeFiles/dephier.exe.dir/build.make:82: recipe for target 'CMakeFiles/dephier.exe.dir/src/main.cpp.o' failed
make[2]: *** [CMakeFiles/dephier.exe.dir/src/main.cpp.o] Error 1
CMakeFiles/Makefile2:116: recipe for target 'CMakeFiles/dephier.exe.dir/all' failed
make[1]: *** [CMakeFiles/dephier.exe.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
CMakeFiles/dephier_paper_tests.exe.dir/build.make:82: recipe for target 'CMakeFiles/dephier_paper_tests.exe.dir/src/dephier_paper_tests.cpp.o' failed
make[2]: *** [CMakeFiles/dephier_paper_tests.exe.dir/src/dephier_paper_tests.cpp.o] Error 1
CMakeFiles/Makefile2:143: recipe for target 'CMakeFiles/dephier_paper_tests.exe.dir/all' failed
make[1]: *** [CMakeFiles/dephier_paper_tests.exe.dir/all] Error 2
Makefile:149: recipe for target 'all' failed
make: *** [all] Error 2

Carving

Consider adding a feature that will allow carving based on the degree of depression nesting, etc.

Compiling error: mising '../common/netcdf.hpp'

See the following error message when running the make command:

make: *** No rule to make target '../common/netcdf.hpp', needed by 'a.out'. Stop.