Automatic parallelization of Python/NumPy, C, and C++ codes on Linux and MacOSX
Home Page: http://www.bh107.org
License: Apache License 2.0
Automatic acceleration of array operations in Python/NumPy, C, and C++ targeting multi-core CPUs and GP-GPUs.
The documentation is available at www.bh107.org.
I'm trying to build Bohrium manually on OSX 10.10, and getting the following errors:
[ 76%] Building CXX object vem/cluster/CMakeFiles/bh_vem_cluster.dir/pgrid.cpp.o
/Users/bogdan/wb/repos/bohrium/vem/cluster/pgrid.cpp:87:9: error: unknown type name 'cpu_set_t'
cpu_set_t cpuset;
^
/Users/bogdan/wb/repos/bohrium/vem/cluster/pgrid.cpp:91:37: error: use of undeclared identifier
'cpu_set_t'; did you mean 'cpuset'?
sched_setaffinity(0, sizeof(cpu_set_t), &cpuset);
^~~~~~~~~
cpuset
/Users/bogdan/wb/repos/bohrium/vem/cluster/pgrid.cpp:87:19: note: 'cpuset' declared here
cpu_set_t cpuset;
^
/Users/bogdan/wb/repos/bohrium/vem/cluster/pgrid.cpp:102:37: error: use of undeclared identifier
'sched_getcpu'
pgrid_myrank, hostname, sched_getcpu(), buf);
^
It seems that this structure should be defined in bits/sched.h, but I can't find this file on my system (or any other header with this definition). I followed the instructions from http://bohrium.readthedocs.org/installation/osx.html for manual installation. I'm open to any suggestions on how to debug this further.
Bohrium (365796a) seems to be missing the following functions:
fmax, fmin, amax, amin, nanmin, nanmax
The following code shows the missing feature of fmax:
import bohrium as bh
import numpy as np
np_a = np.arange(10)
np_b = np.arange(10)[::-1]
np_res = np.fmax(np_a, np_b)
print np_res
bh_a = bh.arange(10)
bh_b = bh.arange(10)[::-1]
bh_res = bh.fmax(bh_a, bh_b)
print bh_resRunning the following script, there never allocated space for b in bohrium land, and the result is discarded without being copied back.
import bohrium as bh
import numpy as np
a = bh.array(np.ones(4))
b = np.empty(1)
b[0] = bh.multiply.reduce(a)
print b
del a
del bgenerates the following trace:
0: BH_MULTIPLY_REDUCE OPS=3{
OP0 [ Dims: 1 Start: 0 Shape: 1 Stride: 1 Base=>[ Addr: 0x1b6d490 Type: BH_FLOAT64 #elem: 1 Data: (nil) ]]
OP1 [ Dims: 1 Start: 0 Shape: 4 Stride: 1 Base=>[ Addr: 0x17a30f0 Type: BH_FLOAT64 #elem: 4 Data: 0x7f6c3a8a1000 ]]
OP2 [ CONST(BH_INT64)=0 ]
}
1: BH_FREE OPS=1{
OP0 [ Dims: 1 Start: 0 Shape: 1 Stride: 1 Base=>[ Addr: 0x1b6d490 Type: BH_FLOAT64 #elem: 1 Data: (nil) ]]
}
2: BH_DISCARD OPS=1{
OP0 [ Dims: 1 Start: 0 Shape: 1 Stride: 1 Base=>[ Addr: 0x1b6d490 Type: BH_FLOAT64 #elem: 1 Data: (nil) ]]
}
3: BH_FREE OPS=1{
OP0 [ Dims: 1 Start: 0 Shape: 4 Stride: 1 Base=>[ Addr: 0x17a30f0 Type: BH_FLOAT64 #elem: 4 Data: 0x7f6c3a8a1000 ]]
}
4: BH_DISCARD OPS=1{
OP0 [ Dims: 1 Start: 0 Shape: 4 Stride: 1 Base=>[ Addr: 0x17a30f0 Type: BH_FLOAT64 #elem: 4 Data: 0x7f6c3a8a1000 ]]
}
This is not a bug, but rather a feature proposal.
Currently the build uses Cheetah, the last version of which was released 5 years ago, and which does not support Py3. There are PR's attempting to fix that in their github repo, but they do not seem to get much attention.
I could prepare a PR switching to some more actively maintained templating engine (Mako is my personal favourite, but I do not insist on it) and fixing possible other incompatibilities with Py3, but I would like to ask first if it is actually needed.
The following code:
import bohrium as bh
test_bh = bh.arange(2)
test_bh[test_bh >= 2]Gives the following error:
/home/Tor/bitbucket/masterthesis/bohrium_tests/zero_length_indexing.py:3: RuntimeWarning: Encountering an operation not supported by Bohrium. It will be handled by the original NumPy.
test_bh[test_bh >= 2]
RuntimeError: The Array Data Protection could not mummap the data region: 0x27514d0 (size: 0). Returned error code by mmap: Invalid argument.
Traceback (most recent call last):
File "/home/Tor/bitbucket/masterthesis/bohrium_tests/zero_length_indexing.py", line 3, in
test_bh[test_bh >= 2]
SystemError: error return without exception set
The following code segfaults
import numpy as np
import bohrium as bh
for module in [np, bh]:
print "Using: {0}".format(module.__name__)
a = module.array([1,0,1,0])
b = module.array([1,1,0,0])
print module.add(a, b)
print module.add.reduce([a, b], axis=0)The bh.add.reduce([a, b], axis=0) line is what causes it to break.
The following code:
import numpy as np
import bohrium as bh
a = bh.arange(10000, dtype=np.int64)
b = np.empty(10000, dtype=np.int64)
print a.dtype, b.dtype
b[:] = a[:]
print "Done"Gives segementation fault.
The following code shows this:
import timeit
print "Bohrium: ", timeit.timeit(
'size = 7 ** 2;'
'bh_array = bh.arange(size);'
'[bh_array[i] for i in xrange(size)]',
setup='import bohrium as bh', number=1000)
print "Numpy: ", timeit.timeit(
'size = 7 ** 2;'
'np_array = np.arange(size);'
'[np_array[i] for i in xrange(size)]',
setup='import numpy as np', number=1000)The result on my machine is:
Bohrium: 36.3068239689
Numpy: 0.00905394554138
According to the doc http://bohrium.readthedocs.org/installation/linux.html#install-from-source-package
MPI is optional. But cmake wil fail if MPI is not available:
-- Could NOT find MPI_C (missing: MPI_C_LIBRARIES MPI_C_INCLUDE_PATH)
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIBRARIES MPI_CXX_INCLUDE_PATH)
CMake Error at vem/cluster/CMakeLists.txt:13 (message):
MPI not found! Set VEM_CLUSTER=OFF or install MPI.
Would be nice if it really was optional. Otherwise it should be listed as a required package.
python numpytest.py
*** Testing the equivalency of Bohrium-NumPy and NumPy ***
... (Lots of testing) ...
Testing test_reduce.py/reduce_prod/prod
: In function ‘KRN_d389407b_b6524c04_9280752f_88e6bd0f’:
:96:8: error: ‘opd2_first’ undeclared (first use in this function)
:96:8: note: each undeclared identifier is reported only once for each function it appears in
pclose(): Success
pclose() failed.
Engine::execute(...) == Compilation failed.
terminate called after throwing an instance of 'std::runtime_error'
what(): Err: Runtime::execute() child->execute() failed: BH_ERROR
Aborted
I am running Fedora Core 21, 64-bit.
Hi, I have OSX 10.9 and
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 6.0 (clang-600.0.57) (based on LLVM 3.5svn)
Target: x86_64-apple-darwin13.4.0
Thread model: posix
Unfortunately, the simple brew install doesn't work, complaining SHA1 mismatch. So i did a manual compile.
However, during execution got this.
AttributeError: 'module' object has no attribute 'genfromtxt'
!! Trying to bind but compiled without required library !!
!! Install hwloc and re-compile !!
!! OR !!
!! Disable binding by setting ENV_VAR: BH_VE_CPU_BIND=0 !!
I got the no attibute part resolved by using original numpy to create the array and passing the array to bohrium to calculate.
I did have hwloc installed via homebrew.
And I did get an error at the end
[1] 30772 bus error python markovchaingpu.py
The following code:
import bohrium as bh
bh_ara = bh.arange(25).reshape(5, 5)
bh_idx = bh.arange(2)
print bh_ara.T[:, bh_idx]Gives segmentation fault and the following error:
/home/ulrik/BitBucket/masterthesis/Playground/bla.py:4: RuntimeWarning: Encountering an operation not supported by Bohrium. It will be handled by the original NumPy.
print bh_ara.T[:, bh_idx]
Traceback (most recent call last):
File "ndarray.pyx", line 239, in ndarray.get_bhc_data_pointer (/home/ulrik/git/bohrium/build/bridge/npbackend/ndarray.c:4328)
File "ndarray.pyx", line 193, in ndarray.get_bhc (/home/ulrik/git/bohrium/build/bridge/npbackend/ndarray.c:3453)
File "ndarray.pyx", line 112, in ndarray.new_bhc_base (/home/ulrik/git/bohrium/build/bridge/npbackend/ndarray.c:2272)
ValueError: For now Bohrium only supports C-style arrays
There are a few warnings @madsbk when building with CMAKE_BUILD_TYPE=Debug.
There are ALOT of warnings when building with CMAKE_BUILD_TYPE=Release. @safl and @tblum and @madsbk
Could we clean this up? I think we should get to where all build types build with -Wall -Werror flags set.
Pretty please with sugar on top :)
Speaking of flags: I thought we moved to -std=c++11? It is still -std=gnu++0x, which is depricated
The following code gives the correct result in numpy, but not in Bohrium (cfcd17f):
import numpy as np
a = map(np.array, range(10))
print np.array(a)
print
import bohrium as bh
a = map(bh.array, range(10))
print bh.array(a)
printThis will give the following result (or something similar):
[0 1 2 3 4 5 6 7 8 9]
[139745276506112 139745276469248 139745276461056 139745276452864
139745276444672 139745276436480 139745276428288 139745276420096
139745276411904 139745276403712]Error message:
Cannot get scalar-value of kp_operand with KP_ETYPE=[KP_COMPLEX64|KP_COMPLEX128|KP_PAIRLL]
When creating bool arrays in Bohrium they contain False when using both ones() and zeroes(). In NumPy, however, using ones() to create a bool array yields True. The following example shows this:
import bohrium as bh
import numpy as np
bh_bool_arr = bh.ones((2, 2), dtype=bh.bool)
print bh_bool_arr
np_bool_arr = np.ones((2, 2), dtype=np.bool)
print np_bool_arrDoes it make any sense to have the fuse cache enabled for fusers that run in O(n), where n is number of instructions?
We have two fusers that run in O(n): singleton and topological.
The the fuse cache hash algorithm runs in O(n). So is there any reason to cache the fused kernels? Writing and reading files on networked file systems are slow.
It would make sense to do some tests, and change the default setting.
@madsbk @safl
We have the helper-tool bhutils.py, however, it does not install as a command.
It would be useful to make it available as a command and while doing so adding a couple of features:
bhutilsbhutils test possibly with multiple granularity bhutils test python, bhutils test cpp or something like that and while looking at testing, document how the various testing systems work.running the following
t_in = np.max(np.array([np.min(txs, axis=0),np.min(tys, axis=0),np.min(tzs, axis=0)]),axis=0)where
np is bohrium
txs, tys, tzs are bohrium arrays shaped (2,20,20,20,400) containing float64s.
yields
Traceback (most recent call last):
File "ndarray.pyx", line 239, in ndarray.get_bhc_data_pointer (/home/koeus/hpc/bohrium/build/bridge/npbackend/ndarray.c:4065)
File "ndarray.pyx", line 171, in ndarray.get_bhc (/home/koeus/hpc/bohrium/build/bridge/npbackend/ndarray.c:2986)
RuntimeWarning: tp_compare didn't return -1 or -2 for exception
the line of code:
test = numpy.array(result)
where result is a bohrium array of shape (30,30), generates the following error:
Traceback (most recent call last):
File "ndarray.pyx", line 239, in ndarray.get_bhc_data_pointer (/home/koeus/hpc/bohrium/build/bridge/npbackend/ndarray.c:4065)
File "ndarray.pyx", line 171, in ndarray.get_bhc (/home/koeus/hpc/bohrium/build/bridge/npbackend/ndarray.c:2986)
RuntimeWarning: tp_compare didn't return -1 or -2 for exception
Segmentation fault (core dumped)
The following code gives segmentation fault:
import bohrium as bh
a = bh.arange(10000).reshape(100, 100)
a[1:-1, 1:-1].ravel()Now that BH_MATMUL exists as a bytecode the extension method implementation should be removed.
I was trying to generate a view containing 0 elements for testing purposes. The following is legal i numpy, but generates an error in bohrium:
import bohrium as np
a = np.ones(9).reshape((3,3))
b = np.ones(9).reshape((3,3))
c = a[3:] + b[3:]
d = a[1:1] + b[1:1]
The following code:
import numpy as np
import bohrium as bh
a = bh.arange(10000, dtype=bh.float64)
b = np.empty(10000, dtype=np.float64)
print a.dtype, b.dtype
b[:] = a[:]
print "Done"Gives segementation fault.
The package libboost-filesystem-dev is missing.
Otherwise the doc for Ubuntu works. Could we add the package to the doc:
http://bohrium.readthedocs.org/installation/linux.html#install-from-source-package
CMake Error at /usr/share/cmake-3.2/Modules/FindBoost.cmake:1201 (message):
Unable to find the requested Boost libraries.
Boost version: 1.55.0
Boost include path: /usr/include
Could not find the following Boost libraries:
boost_filesystem
For all the operations involving complex numbers and two input operands: we should also support one of the input operands being a real number. These are much more efficient to compute, and we save a conversion. So we will gain efficiency in two dimensions.
Example:
"opcode": "BH_ADD",
...
"types": [
[ "BH_BOOL", "BH_BOOL", "BH_BOOL" ],
[ "BH_COMPLEX128", "BH_COMPLEX128", "BH_COMPLEX128" ],
[ "BH_COMPLEX64", "BH_COMPLEX64", "BH_COMPLEX64" ],
[ "BH_COMPLEX128", "BH_COMPLEX128", "BH_FLOAT64" ],
[ "BH_COMPLEX64", "BH_COMPLEX64", "BH_FLOAT32" ],
[ "BH_COMPLEX128", "BH_FLOAT64", "BH_COMPLEX128" ],
[ "BH_COMPLEX64", "BH_FLOAT32", "BH_COMPLEX64" ],
...
This is relevant most for: BH_ADD, BH_SUBTRACT, BH_MULTIPLY, BH_DIVIDE, BH_POWER
The benefits are lesser for, but if we are dong it ... might as well:BH_EQUAL, BH_NOT_EQUAL
It would probably also be relevant for the composite byte code BH_MULTIPLY
OSX 10.10.3
When I run examples, they are executed correctly, but the following warning is shown:
!! Trying to bind but compiled without required library !!
!! Install hwloc and re-compile !!
!! OR !!
!! Disable binding by setting ENV_VAR: BH_VE_CPU_BIND=0 !!
I have hwloc installed (via homebrew) and tried recompiling Bohrium, but it did not remove the warning. There's no mention of hwloc in the cmake output. I can set the environment variable, but I am not sure what hwloc is used for, and whether I'm losing performance by not enabling it.
Building on OSX without Freeglut installed gives:
Building CXX object extmethods/visualizer/CMakeFiles/bh_visualizer.dir/bh_visualizer.cpp.o
In file included from /Users/kenneth/Udvikler/bohrium/extmethods/visualizer/bh_visualizer.cpp:26:
/Users/kenneth/Udvikler/bohrium/extmethods/visualizer/visualizer.hpp:6:10: fatal error:
'GL/freeglut.h' file not found
#include <GL/freeglut.h>
^
1 error generated.
Seems like the GLUT library is not freeglut, hence glut.h needs to be included instead.
Reproduce by running: python -m bohrium idl_init.py --size=16*16 --outputfn=/tmp/hej.npz
From latest benchpress repos.
Example:
python -m bohrium python_numpy/idl_init.py --inputfn=idl_input-float64_512\*512.npz --size=16*16 --dtype=float64 --outputfn=/tmp/huh.npz
python_numpy/idl_init.py - target: bhc, bohrium: True, size: 16*16, elapsed-time: 0.050781
/home/safl/.local/lib/python2.7/site-packages/bohrium/array_create.py:137: RuntimeWarning: Encountering an operation not supported by Bohrium. It will be handled by the original NumPy.
subok=subok, ndmin=ndmin)
The --outputfn triggers it as it calls npbackend.create_array.array(...), the exception-throwing code is around the bottom of the function:
...
if ndarray.check(ary):
ret = ary.copy2numpy()
return numpy.array(ret, dtype=dtype, copy=copy, order=order, \
subok=subok, ndmin=ndmin)
else:
return numpy.array(ary, dtype=dtype, copy=copy, order=order, \
subok=subok, ndmin=ndmin)
...
Some observations
ndarray.check(ary) evaluates to FalseThe problematic tests include test_gauss, test_gauss_20x20, test_jacobi, and test_jacobi_fixed - depending on execution platform.
Observations:
The subprocess started from numpytest.py exits with exit code -11. This can be verified by using subprocess.check_output(cmd) in stead of subprocess.Popen(cmd).communicate()
So multiple instances of bohrium enabled python are running - main process and sub processes. Do we support this in bohrium?
When attaching gdb to the process on the commandline. gdb is attached to the numpytest.py process, end not the subprocess, which is the one failing. How can gdb be attached?
valgrind can be attached to the subprocess by manipulating sys_exec (l227) in numpytets.py. valgrind shows no errors.
gdb can not be attached in this manner. Any ideas why?
The following code:
import bohrium as bh
a = bh.arange(100)
a.shape = (10, 10)
print a.flatten()Gives segmentation fault.
I have the following build options:
-- The following OPTIONAL packages have been found:
-- The following RECOMMENDED packages have been found:
-- The following REQUIRED packages have been found:
-- The following RECOMMENDED packages have not been found:
When I run make on the build files I get the following error:
/home/ulrik/git/bohrium/ve/cpu/utils_mapping_tacs.cpp: In function ‘void bohrium::core::instrs_to_tacs(bh_ir&, std::vector&, bohrium::core::Symbol
Table&)’:
/home/ulrik/git/bohrium/ve/cpu/utils_mapping_tacs.cpp:344:18: error: ‘BH_SIGN’ was not declared in this scope
case BH_SIGN:
^
ve/cpu/CMakeFiles/bh_ve_cpu.dir/build.make:238: recipe for target 've/cpu/CMakeFiles/bh_ve_cpu.dir/utils_mapping_tacs.cpp.o' failed
make[2]: *** [ve/cpu/CMakeFiles/bh_ve_cpu.dir/utils_mapping_tacs.cpp.o] Error 1
CMakeFiles/Makefile2:319: recipe for target 've/cpu/CMakeFiles/bh_ve_cpu.dir/all' failed
make[1]: *** [ve/cpu/CMakeFiles/bh_ve_cpu.dir/all] Error 2
Makefile:117: recipe for target 'all' failed
make: *** [all] Error 2
When using an "invalid" stack definition like the following no warning is given, and it is not clear what setup of components is used. A warning would be nice. But actually I think it should halt with an error.
Example with multiple bcexp:
[stack_gpu]
type = stack
stack_gpu = bcexp
bcexp = dimclean
bcexp = greedy
greedy = node
node = gpu
gpu = cpu
The following code:
import bohrium as bh
test_bh = bh.arange(1)
test_bh[test_bh >= 1]Gives the following error:
RuntimeWarning: Encountering an operation not supported by Bohrium. It will be handled by the original NumPy.
test_bh[test_bh >= 1]
: In function ‘KRN_b51258e4_d32eb96c_655c047f_37c31efe’:
:82:12: error: ‘eidx’ undeclared (first use in this function)
:82:12: note: each undeclared identifier is reported only once for each function it appears in
pclose(): Success
pclose() failed.
Engine::execute(...) == Compilation failed.
terminate called after throwing an instance of 'std::runtime_error'
what(): Err: Runtime::execute() child->execute() failed: BH_ERROR
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/kamp9/.local/lib/python2.7/site-packages/bohrium/__main__.py", line 19, in <module>
execfile(sys.argv[0])
File "tests.py", line 2, in <module>
import scipy.linalg as sp
File "/usr/lib/python2.7/dist-packages/scipy/__init__.py", line 70, in <module>
from numpy import show_config as show_numpy_config
ImportError: cannot import name show_config
The following packages can be added to .travis.yml:
fglrx=2:8.960-0ubuntu1 opencl-headers opencl-dev libgl1-mesa-dev``
It compiles just fine, however, it produces the following at runtime:
$ python ../test/python/numpytest.py --exclude=test_ndstencil.py --exclude=test_benchmarks.py
*** Testing the equivalency of Bohrium-NumPy and NumPy ***
Testing test_accumulate.py/accumulate/cumprod
clGetPlatformIDs
terminate called after throwing an instance of 'std::runtime_error'
what(): Error in the initialization of the VEM.
/home/travis/build.sh: line 41: 13485 Aborted python ../test/python/numpytest.py --exclude=test_ndstencil.py --exclude=test_benchmarks.py
The command "python ../test/python/numpytest.py --exclude=test_ndstencil.py --exclude=test_benchmarks.py" exited with 134.
Which probably makes sense since the resources on travis do not have any GPUs...
But perhaps the GPU-VE could implement some simplified form of execution on CPU?
I guess performance in this setting isn't important. It just needs to be able to run.
Dunno if it makes sense if the testing is not actually done on a GPU?
I don't know of other CI services that actually have GPUs which would be preferable.
But maybe we could roll some integration with MiG or something along those lines...
As the fuse cache needs to incorporate all fielsd used by the fuse filter. We will get the best caching results by making the fuse cache dependent on the fuse model.
The following code:
import bohrium as bh
bh.exp(bh.arange(10))Gives the following error:
Traceback (most recent call last):
File "/home/ulrik/BitBucket/masterthesis/bohrium_tests/bohrium_exp.py", line 2, in
bh.exp(bh.arange(10))
File "ndarray.pyx", line 74, in ndarray.fix_returned_biclass.inner (/home/ulrik/git/bohrium/build/bridge/npbackend/ndarray.c:1652)
File "/home/ulrik/.local/lib64/python2.7/site-packages/bohrium/ufunc.py", line 161, in call
(out_dtype, in_dtype) = _util.type_sig(self.info['name'], args)
File "_util.pyx", line 90, in _util.type_sig (/home/ulrik/git/bohrium/build/bridge/npbackend/_util.c:2211)
TypeError: Cannot determine the correct signature (exp:int64)
np.ones(10,dtype=np.bool)
From ones implementation in npbridge:
a = np.empty((10),dtype=np.bool)
a[...] = a.dtype.type(1)
print a
[False False False False False False False False False False]
trace:
0: BH_IDENTITY OPS=2{
OP0 [ Dims: 1 Start: 0 Shape: 10 Stride: 1 Base=>[ Addr: 0x2ae41c0 Type: BH_BOOL #elem: 10 Data: (nil) ]]
OP1 [ CONST(BH_BOOL)=0c ]
}
Can be done with NumPy:
import numpy as np
m = np.empty(1, )
m[0] = np.float32(0.5)
print np.add.reduce(np.arange(10, dtype=np.float32) * m)
# Output is 0.0, expected 22.5
print np.add.reduce(np.arange(10, dtype=np.float32) * np.float32(0.5))
# Output is 22.5
test.py:
import scipy.linalg
Running command:
python -m bohrium test.py
Gives:
...
from numpy import oldnumeric
ImportError cannot import name oldnumeric
Quick question: Is your project expected to work on 32bit Linux systems? It seems to compile fine for me, but I get this error when I try to do anything with it:
>>> bohrium.zeros(1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "ndarray.pyx", line 74, in ndarray.fix_returned_biclass.inner (bohrium/build/bridge/npbackend/ndarray.c:1651)
File "bohrium/ufunc.py", line 85, in assign
get_base(ary)._data_bhc2np()
TypeError: get_bhc_data_pointer(ary) should return a Python integer that represents a memory address
I added some code that checks if the data in question is a long integer instead and it turns out that it is.
I remember being unable to use this project on another 32bit system a while back, so is that the ultimate cause here?
When compiling Bohrium I get the following error:
CMake Error at bridge/npbackend/cmake_install.cmake:36 (file):
file INSTALL cannot find
"/home/ulrik/git/bohrium/build/bridge/npbackend/lib".
Call Stack (most recent call first):
cmake_install.cmake:70 (include)
Makefile:66: recipe for target 'install' failed
make: *** [install] Error 1
My system configuration is Fedora 64-bit:
Linux Ulrik-LAP-2012 4.1.3-100.fc21.x86_64 #1 SMP Wed Jul 29 18:59:46 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
This bug was fixed in ad19964, but has been reintroduced in 70fb72a.
I have Bohrium git version ae1f62b.
The axis in the Bohrium sum function is not the same as for NumPy when the numbers in the parameter is decreasing. See the following example for reference:
import bohrium as bh
import numpy as np
img_height, img_width, img_depth = 1024, 1024, 8
to_numpy = lambda x: bh.array(x, bohrium=False)
a = bh.random.random((img_height, img_width, img_depth))
b = to_numpy(a)
bh_sum1 = bh.sum(a, axis=(0, 1))
bh_sum2 = bh.sum(a, axis=(1, 0))
bh_sum3 = bh.sum(a, axis=(1, 1))
np_sum1 = np.sum(b, axis=(0, 1))
np_sum2 = np.sum(b, axis=(1, 0))
print "Bohrium shape for axis = (0, 1):", bh_sum1.shape # (8,)
print "Bohrium shape for axis = (1, 0):", bh_sum2.shape # (1024,)
print "Bohrium shape for axis = (1, 1):", bh_sum3.shape # (8,)
print "NumPy shape for axis = (0, 1):", np_sum1.shape # (8,)
print "NumPy shape for axis = (1, 0):", np_sum2.shape # (8,)
print np.allclose(to_numpy(bh_sum1), to_numpy(bh_sum3))
print np.allclose(np_sum1, to_numpy(bh_sum1))
print np.allclose(np_sum1, np_sum2)The problem seems to be in the following code lines in the Bohrium reduce:
else:
tmp1 = self.reduce(ary, axis[0])
axis = [i-1 for i in axis[1:]] # <-- Most likely HERE
tmp2 = self.reduce(tmp1, axis)
if out is not None:
out[...] = tmp2
else:
out = tmp2
return outAmong the unwanted stuff is:
CMakeFiles/
Makefile
_info.py
_util.c
_util.pyx
bhc.i
bhc.py
bhc_wrap.c
build
cmake_install.cmake
ndarray.c
ndarray.pyx
random123.c
random123.pyx
(I am sorry for opening so many issues at once)
I am running a heat equation benchmark from the Benchpress suite: numpy version, bohrium version with witdth 2000 and 40 iterations. The Bohrium time is slightly worse than numpy time: 3.49s against 3.34s. Do you have any suggestions on how I might debug this? Could it be connected to this issue?
Example code:
a = a.astype(np.float64)
Gives warning that the operation is not supported, however this works:
a = np.asarray(a, np.float)
Traceback (most recent call last):
File "bohrium/test/python/numpytest.py", line 373, in <module>
(res1,cmd1) = getattr(cls_inst,mth)(np_arys)
File "/home/travis/build/bh107/bohrium/test/python/test_ndstencil.py", line 18, in test_ndstencil_1D
return self.run(pseudo_arrays)
File "/home/travis/build/bh107/bohrium/test/python/numpytest.py", line 283, in run
res[k] = npzs[k]
File "/usr/lib/python2.7/dist-packages/numpy/lib/npyio.py", line 231, in __getitem__
return format.read_array(value)
File "/usr/lib/python2.7/dist-packages/numpy/lib/format.py", line 440, in read_array
shape, fortran_order, dtype = read_array_header_1_0(fp)
File "/usr/lib/python2.7/dist-packages/numpy/lib/format.py", line 336, in read_array_header_1_0
d = safe_eval(header)
File "/usr/lib/python2.7/dist-packages/numpy/lib/utils.py", line 1132, in safe_eval
import compiler
File "/usr/lib/python2.7/compiler/__init__.py", line 27, in <module>
DeprecationWarning, stacklevel=2)
DeprecationWarning: The compiler package is deprecated and removed in Python 3.x.
The strange thing is that travis is NOT running python3 by default.
Anyways made a quickfix: 3e8c777
We might want a more robust solution?
The following code:
import bohrium as bh
a = bh.arange(100)
a.shape = (10, 10)
print a.flatten()Gives segmentation fault.
This was fixed at some point in time, but now gives segmentation fault yet again (as of commit 6aaa9ea, but might be way before this commit).
S = np.array(numpy.random.random(10).astype(numpy.float32))
d1 = S/np.sqrt(4.0)
Forces d1 to be of float64 data type where as
d1 = S/math.sqrt(4.0)
Keeps d1 as float32. Which it should be.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.