adaptiveparticles / libapr Goto Github PK

View Code? Open in Web Editor NEW

47.0 47.0 14.0 52.88 MB

Library for producing and processing on the Adaptive Particle Representation (APR).

License: Apache License 2.0

CMake 1.37% C++ 77.14% C 0.55% Cuda 19.40% MATLAB 1.54%

adaptive biology image-processing microscopy multi-resolution sampling-methods

libapr's People

Contributors

Stargazers

Watchers

Forkers

yuanhaogong krzysg biodataanalysis psteinb suryanarayanamk msusik gitter-badger tiwezhang litaochen bryantchhun volkerh jimk1983 yurongchen1998 5l1v3r1

libapr's Issues

Compilation issues on Ubuntu/gcc

[ 85%] Building CXX object CMakeFiles/libapr.dir/src/vis/RaytracedObject.cpp.o
In file included from /fastbeast/LibAPR/src/io/hdf5functions_blosc.cpp:11:0:
/fastbeast/LibAPR/src/io/hdf5functions_blosc.h:15:19: fatal error: hdf5.h: No such file or directory
compilation terminated.
CMakeFiles/libapr.dir/build.make:86: recipe for target 'CMakeFiles/libapr.dir/src/io/hdf5functions_blosc.cpp.o' failed
make[2]: *** [CMakeFiles/libapr.dir/src/io/hdf5functions_blosc.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/fastbeast/LibAPR/src/io/blosc_filter.c:18:18: fatal error: hdf5.h: No such file or directory
compilation terminated.
CMakeFiles/libapr.dir/build.make:62: recipe for target 'CMakeFiles/libapr.dir/src/io/blosc_filter.c.o' failed
make[2]: *** [CMakeFiles/libapr.dir/src/io/blosc_filter.c.o] Error 1
[ 85%] Linking C executable bench
[ 85%] Built target bench
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/libapr.dir/all' failed
make[1]: *** [CMakeFiles/libapr.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Update naming conventions in ExtraPartCellData

*Should make the naming consistent and change from depth_max to the newer convention of level.
*Also should migrate some helper functions from APRTimeIO in APR_time develop into the ExtraPartCellData, as it seems this data-structure type is useful in the time context.

On linux getting Java and google test errors, however everything is working fine on Mac osx

Could be I need to update things, but not sure.

CreateSmallSphereTest.APR_INPUT_OUTPUT hangs when compiled with Intel compiler

Compiled with Intel's Parallel Studio 2018u1:

CC="icc" CXX="icc" CXXFLAGS="-O3 -g" cmake -DAPR_TESTS=1 ..
make -j 7 testAPR

Test run with command:
./test/testAPR --gtest_filter=*APR_INPUT_OUTPUT*

Output:

[ RUN      ] CreateSmallSphereTest.APR_INPUT_OUTPUT
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_level.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_type.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_original.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_pc.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_x.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_y.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4
Opening file: [/projects/ppm/gonciarz/AdaptiveParticleRepresentation/test/files/Apr/sphere_120/sphere_z.tif]
getMesh: MeshData: size(Y/X/Z)=120/120/120 vSize:1728000 vCapacity:1728000 elementSize:2
getMesh: ScanlineSize=240 StripSize=8160 NumberOfStrips=4

Seems that it hangs just before/during writing HDF5 file (comparing to output from gcc/clang). This is the only test that fails.

Add Travis CI integration

We'd like to run our tests in an automated fashion, so it'd be nice to set up Travis at some point.

Neighborhood optimization -> shoudl make this optional and set a flag

This is just a change in the creation of the datastructure for creation of the APR from V.

Segmentation fault

Hello,
I got "segmentation fault" error when transforming 2D TIFF image to APR format.
I am using the python wrapper
The output looks like:

Opening file: [path/to/file.tif]
Opening file: [path/to/file.tif]
getMesh: PixelData: size(Y/X/Z)=2160/2560/1 vSize:5529600 vCapacity:5529600 elementSize:2
getMesh: ScanlineSize=5120 StripSize=5120 NumberOfStrips=2160
compute_gradient_magnitude_using_bsplines took 0.110055 seconds
CPU WINDOWS: 1 1 1 3 3 3
Segmentation fault: 11

Any idea why is this happening?

Increase test coverage of iteration operations

Would be great to include some test coverage on:

The random access components of the GenIterator.
The access components of the APRTree.
Find particle cell from spatial point.
Reconstruction Condition test.
Compression Class.

conda-forge package

It would be nice to be able to install this library and the python-bindings via conda-forge.

Merge 2D image support into the main branch

See #104, we should clean up and merge the 2D support into master .

*ideally also extend the test coverage before we do this as well.

*Should also do a branch clean-up

After latest fix pipeline test with GCC is not passing

Fix introduced here: 7053d6d broke something.
On a master branch before fix everything works fine.

When run with clang:

cmake -DAPR_TESTS=ON ..
make -j 7 testAPR ; ./test/testAPR

everything is fine:

[----------] 4 tests from CreateSmallSphereTest (624 ms total)

[----------] Global test environment tear-down
[==========] 4 tests from 1 test case ran. (625 ms total)
[  PASSED  ] 4 tests.

but when running same stuff with GCC:

CC="gcc-7" CXX="g++-7" cmake -DAPR_TESTS=ON ..
make -j 7 testAPR ; ./test/testAPR

It fails:

[==========] 4 tests from 1 test case ran. (790 ms total)
[  PASSED  ] 3 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] CreateSmallSphereTest.APR_PIPELINE

 1 FAILED TEST

Windows mingw static linking error

On develop when compiling the library using mingw64 installed using msys2, I get the following compilation error when -DAPR_BUILD_STATIC_LIB=ON

process_begin: CreateProcess(NULL, lib.exe /OUT:C:/Users/bevanc/LibAPR/build/libapr.a C:/Users/bevanc/LibAPR/build/external/c-blosc/blosc/libblosc.a, ...) failed. make (e=2): The system cannot find the file specified. mingw32-make[2]: *** [CMakeFiles\staticLib.dir\build.make:74: libapr.a] Error 2 mingw32-make[2]: *** Deleting file 'libapr.a' mingw32-make[1]: *** [CMakeFiles\Makefile2:110: CMakeFiles/staticLib.dir/all] Error 2 mingw32-make[1]: *** Waiting for unfinished jobs.... [ 79%] Built target sharedLib mingw32-make: *** [Makefile:94: all] Error 2

APRNumerics::compute_gradient_vector() fails to calculate gradient for last particle

It seems that APRNumerics::compute_gradient_vector() is missing the last particle in the dataset. I can consistently reproduce this with all the datasets I have, and the Java wrappers.

Need to add updated install instruction to Readme

The Readme needs to be slightly updated to include the correct clone recursive and or git submodule init, git submodule update commands

command line arguments ignored for Example_get_apr

when used with head.tif (it does not happen always). Command line to reproduce:
Example_get_apr -i head.tif -rel_error 0.01

Obviously it is overwritten by this code in auto_parameters(...):
https://github.com/cheesema/LibAPR/blob/2dddea899c668e12c770d1473c2fb4d95c0955be/src/algorithm/APRConverter.hpp#L563-L574

Before naive fix I think we need to discuss how to handle auto_parameters:

should they be turned off when user provide any parameter like error/threshold etc.
should they change only not provided (will it work correctly?)
any other approach? like command line flag to actually use auto parameters explicitly?

Master Failing

python fix caused master now to have a compilation error.

APRRaycaster removed due to compilation from develop

Due to include issues with APRRaycaster, I have removed it to allow pushing of recent changes into develop. We need to figure this out and then add back.

examples are not built by default

In the instruction there is a statement:

This will create the libapr.so library in the build directory, as well as all of the examples.

But When I follow the instruction, the examples were not built.
By inspecting the file "CMakeList.txt" I saw the switch is "OFF" by default.

option(APR_BUILD_EXAMPLES "Build APR examples" OFF)

Shall it be changed to "ON"?

Possible Bug in Neighbour Cells

Seems like there could be some neighbours that are being missed, due to a no-neighbour flag being set incorreclty. (+y neighbours)

Python `init.py` file missing?

Hi,

I'm trying to get the python samples to work, but I am wondering whether some python code is missing from the repository. Specifically I assume I would need something like an __init.py__ to load pyApr as a module.

I built libapr with cmake using the option -DAPR_BUILD_PYTHON_WRAPPERS=ON. I'm not sure whether the cmake step should produce the .py files that I need to import pyAPR.
I can see that I have a pyApr.so in the build folder and I'm adding that folder to the system path.
However, I can't import pyApr.

For reference: I've been building this inside the following docker environment:
https://github.com/VolkerH/MyDockerStuff/tree/master/my_libapr

the build folder looks as follows:

root@9d00051e6719:/srv/LibAPR/build# tree
.
|-- CMakeCache.txt
|-- CMakeFiles
|   |-- 3.10.2
|   |   |-- CMakeCCompiler.cmake
|   |   |-- CMakeCXXCompiler.cmake
|   |   |-- CMakeDetermineCompilerABI_C.bin
|   |   |-- CMakeDetermineCompilerABI_CXX.bin
|   |   |-- CMakeSystem.cmake
|   |   |-- CompilerIdC
|   |   |   |-- CMakeCCompilerId.c
|   |   |   |-- a.out
|   |   |   `-- tmp
|   |   `-- CompilerIdCXX
|   |       |-- CMakeCXXCompilerId.cpp
|   |       |-- a.out
|   |       `-- tmp
|   |-- CMakeDirectoryInformation.cmake
|   |-- CMakeOutput.log
|   |-- CMakeTmp
|   |-- FindOpenMP
|   |   |-- OpenMPCheckVersion.c
|   |   |-- OpenMPCheckVersion.cpp
|   |   |-- OpenMPTryFlag.c
|   |   |-- OpenMPTryFlag.cpp
|   |   |-- ompver_C.bin
|   |   `-- ompver_CXX.bin
|   |-- Makefile.cmake
|   |-- Makefile2
|   |-- TargetDirectories.txt
|   |-- aprObjLib.dir
|   |   |-- C.includecache
|   |   |-- CXX.includecache
|   |   |-- DependInfo.cmake
|   |   |-- build.make
|   |   |-- cmake_clean.cmake
|   |   |-- depend.internal
|   |   |-- depend.make
|   |   |-- flags.make
|   |   |-- progress.make
|   |   `-- src
|   |       `-- io
|   |           |-- blosc_filter.c.o
|   |           `-- hdf5functions_blosc.cpp.o
|   |-- cmake.check_cache
|   |-- feature_tests.bin
|   |-- feature_tests.c
|   |-- feature_tests.cxx
|   |-- hdf5
|   |   |-- CMakeFiles
|   |   |   `-- CMakeTmp
|   |   `-- cmake_hdf5_test.c
|   |-- progress.marks
|   |-- pyApr.dir
|   |   |-- CXX.includecache
|   |   |-- DependInfo.cmake
|   |   |-- build.make
|   |   |-- cmake_clean.cmake
|   |   |-- depend.internal
|   |   |-- depend.make
|   |   |-- flags.make
|   |   |-- link.txt
|   |   |-- progress.make
|   |   `-- src
|   |       `-- wrapper
|   |           `-- pythonBind.cpp.o
|   `-- sharedLib.dir
|       |-- DependInfo.cmake
|       |-- build.make
|       |-- cmake_clean.cmake
|       |-- depend.internal
|       |-- depend.make
|       |-- flags.make
|       |-- link.txt
|       `-- progress.make
|-- ConfigAPR.h
|-- Makefile
|-- cmake_install.cmake
|-- external
|   |-- c-blosc
|   |   |-- CMakeFiles
|   |   |   |-- CMakeDirectoryInformation.cmake
|   |   |   `-- progress.marks
|   |   |-- Makefile
|   |   |-- blosc
|   |   |   |-- CMakeFiles
|   |   |   |   |-- CMakeDirectoryInformation.cmake
|   |   |   |   |-- blosc_static.dir
|   |   |   |   |   |-- C.includecache
|   |   |   |   |   |-- CXX.includecache
|   |   |   |   |   |-- DependInfo.cmake
|   |   |   |   |   |-- __
|   |   |   |   |   |   `-- internal-complibs
|   |   |   |   |   |       |-- lizard-1.0
|   |   |   |   |   |       |   |-- lizard_compress.c.o
|   |   |   |   |   |       |   |-- lizard_decompress.c.o
|   |   |   |   |   |       |   `-- lizard_frame.c.o
|   |   |   |   |   |       |-- lz4-1.7.5
|   |   |   |   |   |       |   |-- lz4.c.o
|   |   |   |   |   |       |   `-- lz4hc.c.o
|   |   |   |   |   |       |-- snappy-1.1.1
|   |   |   |   |   |       |   |-- snappy-c.cc.o
|   |   |   |   |   |       |   |-- snappy-sinksource.cc.o
|   |   |   |   |   |       |   |-- snappy-stubs-internal.cc.o
|   |   |   |   |   |       |   `-- snappy.cc.o
|   |   |   |   |   |       `-- zstd-1.3.0
|   |   |   |   |   |           |-- common
|   |   |   |   |   |           |   |-- entropy_common.c.o
|   |   |   |   |   |           |   |-- error_private.c.o
|   |   |   |   |   |           |   |-- fse_decompress.c.o
|   |   |   |   |   |           |   |-- pool.c.o
|   |   |   |   |   |           |   |-- threading.c.o
|   |   |   |   |   |           |   |-- xxhash.c.o
|   |   |   |   |   |           |   `-- zstd_common.c.o
|   |   |   |   |   |           |-- compress
|   |   |   |   |   |           |   |-- fse_compress.c.o
|   |   |   |   |   |           |   |-- huf_compress.c.o
|   |   |   |   |   |           |   |-- zstd_compress.c.o
|   |   |   |   |   |           |   `-- zstdmt_compress.c.o
|   |   |   |   |   |           `-- decompress
|   |   |   |   |   |               |-- huf_decompress.c.o
|   |   |   |   |   |               `-- zstd_decompress.c.o
|   |   |   |   |   |-- bitshuffle-avx2.c.o
|   |   |   |   |   |-- bitshuffle-generic.c.o
|   |   |   |   |   |-- bitshuffle-sse2.c.o
|   |   |   |   |   |-- blosc.c.o
|   |   |   |   |   |-- blosclz.c.o
|   |   |   |   |   |-- build.make
|   |   |   |   |   |-- cmake_clean.cmake
|   |   |   |   |   |-- cmake_clean_target.cmake
|   |   |   |   |   |-- depend.internal
|   |   |   |   |   |-- depend.make
|   |   |   |   |   |-- flags.make
|   |   |   |   |   |-- link.txt
|   |   |   |   |   |-- progress.make
|   |   |   |   |   |-- shuffle-avx2.c.o
|   |   |   |   |   |-- shuffle-generic.c.o
|   |   |   |   |   |-- shuffle-sse2.c.o
|   |   |   |   |   `-- shuffle.c.o
|   |   |   |   `-- progress.marks
|   |   |   |-- Makefile
|   |   |   |-- cmake_install.cmake
|   |   |   |-- config.h
|   |   |   `-- libblosc.a
|   |   `-- cmake_install.cmake
|   `-- pybind11
|       |-- CMakeFiles
|       |   |-- CMakeDirectoryInformation.cmake
|       |   `-- progress.marks
|       |-- Makefile
|       `-- cmake_install.cmake
|-- libapr.so -> libapr.so.1
|-- libapr.so.1 -> libapr.so.1.1.0
|-- libapr.so.1.1.0
`-- pyApr.so

35 directories, 119 files
root@9d00051e6719:/srv/LibAPR/build#

Add Doxygen or similar

It would be great to add Doxygen or similar for auto-generation of documentation and start a push towards standardisation of the style and comment/description data as to work with such a system.

python pixel->apr->pixel roundtrip results in transposed image

Hi,

just did a short roundtrip from a pixel image to APR back to pixel image and the result seems to be tranposed:

import numpy as np
import pyApr 

x = np.zeros((44,55,66),np.uint16)
apr = pyApr.AprShort() 
apr.get_apr_from_array(x)
y = np.array(apr.reconstruct(), copy = False)

print(x.shape)
print(y.shape)
# (44, 55, 66)
# (55, 66, 44)

Namechange MeshData -> PixelData

What do you guys think @skalarproduktraum this would make you happy?

@krzysg ?

APRTest failing on master

i'll look into it.

Maximum and Minimum over all particles and by level

python bindings crash at import (linux/mac)

Hi,

First, thanks for the great package, which I would love to use from within python.
I followed the install instructions for python support (cmake -DAPR_BUILD_PYTHON_WRAPPERS=ON ..) and everything builds fine. However if I try to import the built python module (pyApr.so) via import pyApr it crashes at (1) Mac and (2) Linux with the following errors:

(1) OSX 10.12.6, clang 8.0.0, python 3.6.4

Termination Signal:    Illegal instruction: 4
dyld: in dlopen()
BUG IN LIBTRACE: MH not found
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_trace.dylib         	0x00007fff9c24de97 _os_trace_image_was_unloaded + 271
1   dyld                          	0x000000010f2cacb2 dyld::removeImage(ImageLoader*) + 343

(2) Ubuntu 16.04, gcc 6.4.0, python 3.5.2

ImportError: /home/mweigert/cpp/LibAPR/build/pyApr.so: undefined symbol: _Py_ZeroStruct

get_parts_from_img

We need to bring back from the dead one of the functions here. There was also a function that then sampled the particles from an input image from the nearest pixel, currently, it is only supporting for a tree of down-sampled images.

It was probably deleted a week or so ago, I should be able to find it, I guess I should find an example somewhere to display its functionality, as I have found it quite useful in the past.

Java wrappers does not built anymore (-DAPR_BUILD_JAVA_WRAPPERS=ON)

Example error output:

make[1]: *** Waiting for unfinished jobs....
In file included from /Users/krzysg/APR/AdaptiveParticleRepresentation/src/main/java/de/mpicbg/mosaic/apr/libaprJAVA_wrap.cxx:237:
/usr/local/Cellar/llvm/6.0.0/include/c++/v1/vector:1359:9: error: no matching member function for call to 'assign'
        assign(__x.__begin_, __x.__end_);

I have no idea when it started to failing (I have checked several branches going back in time like 2 weeks but fails on all of them).

So the main things that should be addressed here are:

obviously we need fix (or decision to abandon swig and go with JavaCPP)
we should somehow add build of java wrappers to travis to avoid such situations in future

Neighbour iteration comparison

For comparison and benchmark of performance compared to alternative data structures in OpenFPM.

Need some candidate datasets and a way to write them for reading into OpenFPM.

Code exists, just needs re-organization.

Additional (more intuitive naming) looping functions for iterators

Current looping structures looks on the lines of:

        for (unsigned int level = apr_iterator.level_min(); level <= apr_iterator.level_max(); ++level) {
            for (int z = 0; z < apr_iterator.spatial_index_z_max(level); z++) {
                for (int x = 0; x < apr_iterator.spatial_index_x_max(level); ++x) {
                    for (apr_iterator.set_new_lzx(level, z, x); apr_iterator.global_index() < apr_iterator.end_index;
                         apr_iterator.set_iterator_to_particle_next_particle()) {

Thought about changing the loop interface to also allow the form:

        for (unsigned int level = apr_iterator.level_min(); level <= apr_iterator.level_max(); ++level) {

            for (int z = 0; z < apr_iterator.z_max(level); z++) {
                for (int x = 0; x < apr_iterator.x_max(level); ++x) {
                    for (apr_iterator.begin(level, z, x); apr_iterator < apr_iterator.end();
                         apr_iterator++) {

@skalarproduktraum @krzysg @joeljonsson what do you guys think?

Minimal Version for Release

Over Christmas thought about what the first targets should be.

The basic functionality it would be nice to be able to have with documentation would be:

Convert image to APR (with default parameter, and parameter selection)
Example providing two different reconstruction to Image file (smooth, pc)
Example showing looping over all particles and accessing the level, type and intensity, also showing neighbor access, creation of new datasets with the APR structure.
Example showing calculation of gradient using the APR.
Example writing to full format that is readable in Paraview
Example doing the APR ray-cast

Demo datasets:
include 2-3 example APR datasets, two real and maybe two spheres of different sizes.

All of this exists, would just be a matter of cleaning up, checking, and providing some more documentation.

Tests no longer compiling

On master I am unable to compile the tests. Due to duplicate symbols.

Compression with prediction step seems to introduce over-flow

Currently compression with setting 2, can lead to an integer over-flow somewhere.

Removal of redundant structures from APRAccess

There are a few helper caches in APRAccess that are redundant, and not ever used anywhere, and could be removed to make the reader/write classes nicer.

In particular:

 std::vector<uint64_t> global_index_by_level_begin;
    std::vector<uint64_t> global_index_by_level_end;
    std::vector<std::vector<uint64_t>> global_index_by_level_and_z_begin;
    std::vector<std::vector<uint64_t>> global_index_by_level_and_z_end;

All information required for these can already be found in:

std::vector<std::vector<uint64_t>> global_index_by_level_and_zx_end;

This is also currently how the iterators get this information. Further, the definitions of the global_index_by_level_begin & end follow a different convention, requiring either a removal or re-write.

Removal of the structure would then require the update of the following functions in GenIterator:

  uint64_t particles_level_begin(const uint16_t& level_);
  uint64_t particles_level_end(const uint16_t& level_);

    uint64_t particles_z_begin(const uint16_t& level_,const uint64_t& z_);
    uint64_t particles_z_end(const uint16_t& level_,const uint64_t& z_);

Bug in map_inplace in iterator example on sphere_400 example

working on it

I am getting seg faults when running multiple get_apr commands in sequence

I have a loop of multiple calls to the get_apr, unfortunately, its on my benchmarks, in Benchmark_real_datasets

force_load not supported on MS architectures

When building the latest head on MSVC 2017, I get errors:

LINK : warning LNK4044: unrecognized option '/Wl,-force_load,D:/tmp-debug-MSVC-Generic-7-x64-cl19.11.25547/LibAPR/external/c-blosc/blosc/libblosc.lib'; ignored
   Creating library apr.lib and object apr.exp
blosc_filter.c.obj : error LNK2019: unresolved external symbol blosc_compress referenced in function blosc_filter
blosc_filter.c.obj : error LNK2019: unresolved external symbol blosc_decompress referenced in function blosc_filter
blosc_filter.c.obj : error LNK2019: unresolved external symbol blosc_set_compressor referenced in function blosc_filter
blosc_filter.c.obj : error LNK2019: unresolved external symbol blosc_compcode_to_compname referenced in function blosc_filter
blosc_filter.c.obj : error LNK2019: unresolved external symbol blosc_list_compressors referenced in function blosc_filter
blosc_filter.c.obj : error LNK2019: unresolved external symbol blosc_cbuffer_sizes referenced in function blosc_filter
apr.dll : fatal error LNK1120: 6 unresolved externals

Is it correct that linker option force_load is not supported on MS architectures? I found https://stackoverflow.com/questions/25038974/force-load-linker-flag-for-other-platforms

Add option to write intermediate steps to tif

Would be nice for debugging / understanding to be able to visualise gradient, scale etc.

Example build error in Ubuntu 14.04

I could successfully install LibAPR, but when I try to build the example file I met some problem.
My system is Ubuntu 14.04
CMake version: 3.6.3
GCC version: 5.2.0
When I try to run the "cmake-build.sh "script in the LibAPR folder, I got the following error:

So I try to install the snappy lib in https://github.com/google/snappy
However, it couldn't fix my problem.
I also try to build example in the examples/build folder, still didn't find a way out. I got the following error.

It seems a Cpp standard error, so I changed the CMakeLists.txt file in the examples folder by adding "set (CMAKE_CXX_STANDARD 14)". But it still get the error as following:

I am not very familiar with Cpp, can you gave me some advice?
I would be very grateful for your help！

calc_sat_mean_z method calculates wrong values when used offset is close to img dimension

I have noticed that when comparing to GPU version when z-direction was equal to 7 and offset equal to 3 in the output of calc_sat_mean_z there is a lot of 0's. When z-dir length is big enough (like 13) everything seems to be correct.

Remove pybind11 from the submodules

Give we have externalised the python wrappers, we no longer need pybind11 in the submodules, nor do we need any python logic in the cmake.

Is this correct @krzysg @joeljonsson ?

If so we can then go ahead and remove these.

Change to https://github.com/jkriege2/TinyTIFF

for tiff IO?

Aparentlly it is like 100 times faster.. for larger images reading in the image takes about 20-30% of the total time.

APRTest

Not passing on Linux - APR Pipeline,

Sporadic, exit memory error using MallocCheckHeapEach flag

There is still a bug, with MallocCheckHeapEach, from an Hdf5 garbage collector, that sporadically causes an error on exit with the flag enabled. Can't figure out whats causing it.

Get it every 2-3 run Example_apr_iterate on master, running on osx.

Valgrind has no issues, excluding the pointers in the register_blosc(), function.

windows build error

I met a link error when I was trying to build LibAPR.
My system is windows 10
CMake version is 3.11.1
Compiler is Intel 18.0.2.20180210
when I ran

cmake -G "Visual Studio 15 2017 Win64" -DTIFF_INCLUDE_DIR="C:/Program Files/tiff/include" -DTIFF_LIBRARY="C:/Program Files/tiff/lib/tiff.lib " -DHDF5_ROOT="C:/Program Files/HDF_Group/HDF5/1.10.2" -DAPR_PREFER_EXTERNAL_BLOSC="OFF" -T "Intel C++ Compiler 18.0" ..

I got

-- The CXX compiler identification is Intel 18.0.2.20180210
-- Check for working C compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2018/windows/bin/intel64/icl.exe
-- Check for working C compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2018/windows/bin/intel64/icl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2018/windows/bin/intel64/icl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/IntelSWTools/compilers_and_libraries_2018/windows/bin/intel64/icl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Configuring for APR version: 1.0.1
-- Found HDF5: C:/Program Files/HDF_Group/HDF5/1.10.2/lib/hdf5.lib (found version "1.10.2")
-- Found ZLIB: C:/Users/Server 36/Anaconda3/Library/lib/z.lib (found version "1.2.8")
-- Found TIFF: C:/Program Files/tiff/lib/tiff.lib (found version "4.0.9")
-- Found OpenMP_C: -Qopenmp (found version "5.0")
-- Found OpenMP_CXX: -Qopenmp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- APR: blosc not found, using internal blosc
Configuring for Blosc version: 1.11.2
-- Using LZ4 internal sources.
-- Using Snappy internal sources.
-- No Zstd library found.  Using internal sources.
-- Building for system processor AMD64
-- Adding run-time support for SSE2
-- Looking for pthread.h
-- Looking for pthread.h - not found
-- Found Threads: TRUE
-- APR: Install library in [C:/Program Files/LibAPR]
-- APR: Building examples
-- Configuring done
-- Generating done
-- Build files have been written to: E:/LibAPR/build

And then I ran cmake --build . --config Debug
Then the error came up:

  blosclz.c
  shuffle-generic.c
  bitshuffle-generic.c
  shuffle-sse2.c
  bitshuffle-sse2.c
  lz4.c
E:\LibAPR\external\c-blosc\internal-complibs\lz4-1.7.5\lz4.c(184): error : expected a type specifier [E:\LibAPR\build\e
xternal\c-blosc\blosc\blosc_static.vcxproj]
    typedef union { U16 u16; U32 u32; reg_t uArch; } __attribute__((packed)) unalign;
                                                                   ^

E:\LibAPR\external\c-blosc\internal-complibs\lz4-1.7.5\lz4.c(184): error : expected a ";" [E:\LibAPR\build\external\c-b
losc\blosc\blosc_static.vcxproj]
    typedef union { U16 u16; U32 u32; reg_t uArch; } __attribute__((packed)) unalign;
                                                                             ^

E:\LibAPR\external\c-blosc\internal-complibs\lz4-1.7.5\lz4.c(186): error : identifier "unalign" is undefined [E:\LibAPR
\build\external\c-blosc\blosc\blosc_static.vcxproj]`
And also error as followed:
`CustomBuild:
  Building Custom Rule E:/LibAPR/CMakeLists.txt
  CMake does not need to re-run because E:/LibAPR/build/CMakeFiles/generate.stamp is up-to-date.
Lib:
  C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2018\windows\bin\Intel64\xilib.exe -qm64 /OUT:"E:\LibAPR\
  build\Debug\apr.lib" /NOLOGO /MACHINE:X64 E:\LibAPR\build\aprObjLib.dir\Debug\blosc_filter.obj
  E:\LibAPR\build\aprObjLib.dir\Debug\hdf5functions_blosc.obj
  E:\LibAPR\build\aprObjLib.dir\Debug\APRRaycaster.obj
  E:\LibAPR\build\aprObjLib.dir\Debug\Camera.obj
  E:\LibAPR\build\aprObjLib.dir\Debug\Object.obj
  E:\LibAPR\build\aprObjLib.dir\Debug\RaytracedObject.obj  /machine:x64 ""$"<TARGET_FILE:blosc_static>"
LINK : : error LNK1104: can't open file“$<TARGET_FILE:blosc_static>” [E:\LibAPR\build\staticLib.vcxproj]

It looks like a blosc link error, but I don't know how to fix it. I tried to compile the blosc library in https://github.com/Blosc/c-blosc and add a '-DAPR_PREFER_EXTERNAL_BL
OSC="ON"' in the CMake commond, but it didn't work.
Would you please help me？

Edit by @skalarproduktraum: Improved formatting for better readability

Choice of license

What license shall we choose for the library?

GPL3 - 'infectious', but will keep derivative works open-source, cannot be used in commercial software then
LGPL3 - same as GPL, but can be linked against in commercial software
MIT - do whatever, no guarantees, keep the copyright intact
Apache License 2.0 - do whatever, but don't sue me. Provides some protection against litigation, also from patent trolls trying to profit off the software

My suggestion would be LGPL3 or Apache.