intel/pymic

pymic's Issues

compilation for MIC does not pick up the correct libraries openMP

I'm using anaconda, when compiling devito for host looks at ~/anaconda2/lib for libiomp5.so
when compiling devito for MIC it does the same.

Instead it should ask for directory containing Intel MIC libraries normally found under /opt/intel/compilers_and_libraries/something/something/lib/

Can not load library compiled with mkl

pyMIC can load library compiled without mkl normally. But when it come to library that compile link to mkl, there are some errors. This error happen when I run examples/dgemm/dgemm.py

Detailes :

PYMIC: debug level set to 1000
PYMIC: tracing is disabled
PYMIC: found 2 device(s)
PYMIC: created stream 0x1219360 for device 0
PYMIC: created stream 0x1229400 for device 1
PYMIC: searching for liboffload_array.so in 
PYMIC:     looking for liboffload_array.so in /usr/lib/python2.7/site-packages/pymic
PYMIC: loading '/usr/lib/python2.7/site-packages/pymic/liboffload_array.so' on device 0
PYMIC: successfully loaded '/usr/lib/python2.7/site-packages/pymic/liboffload_array.so' on device 0 with handle 0x7fa7f4001220
PYMIC: searching for liboffload_array.so in 
PYMIC:     looking for liboffload_array.so in /usr/lib/python2.7/site-packages/pymic
PYMIC: loading '/usr/lib/python2.7/site-packages/pymic/liboffload_array.so' on device 1
PYMIC: successfully loaded 'usr/lib/python2.7/site-packages/pymic/liboffload_array.so' on device 1 with handle 0x7f0b50001220
PYMIC: starting initialization of the offload infrastructure
PYMIC: loading supporting pyMIC libraries on all devices
PYMIC: searching for libdgemm.so in 
PYMIC:     looking for libdgemm.so in /usr/lib/python2.7/site-packages/pymic
PYMIC:     looking for libdgemm.so in 
PYMIC: loading 'libdgemm.so' on device 0
caught!
Traceback (most recent call last):
  File "dgemm.py", line 41, in <module>
    library = device.load_library("libdgemm.so")
  File "/usr/lib/python2.7/site-packages/pymic/_tracing.py", line 128, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/pymic/offload_device.py", line 168, in load_library
    return OffloadLibrary(libraries[0], device=self)
  File "/usr/lib/python2.7/site-packages/pymic/offload_library.py", line 125, in __init__
    filename)
  File "pymic_libxstream.pyx", line 238, in pymic.pymic_libxstream.pymic_library_load (src/pymic_libxstream.cpp:3943)
  File "pymic_libxstream.pyx", line 231, in pymic.pymic_libxstream._c_pymic_library_load (src/pymic_libxstream.cpp:3757)
pymic.offload_error.OffloadError: Could not load library 'libdgemm.so' on device 0
offload error: cannot unload library from the device 0 (error code 14)

Testbed:

Intel Xeon Phi 7120P
Redhat 7.2
Intel® MPSS 3.7.2

pyMIC dgemm performance?

I'm getting around 100 GFLOPS using the dgemm.py example for 4096x4096 matrices vs ~300 GFLOPS reported in http://www.dlr.de/sc/Portaldata/15/Resources/dokumente/pyhpc2014/submissions/pyhpc2014_submission_8.pdf.

any ideas on why?

Thanks

pyMIC + mlpack

We are trying to compile some code (a modified mlpack knn_example.cpp) that uses the mlpack and Armadillo c++ libraries . Compilation is successful but when running the pymic code we are getting an error pymic.offload_error.OffloadError: Could not load library 'knn.so' on device 0

The modified c++ code is:

extern "C" {
    #include <Python.h>
    #include <numpy/arrayobject.h>
}

#include <mlpack/core.hpp>
#include <mlpack/methods/neighbor_search/neighbor_search.hpp>
#include <pymic_kernel.h>

using namespace mlpack;
using namespace mlpack::neighbor; // NeighborSearch and NearestNeighborSort
using namespace mlpack::metric; // ManhattanDistance

PYMIC_KERNEL
void calc_knn(double *arr, double *distarr)
{
    // Armadillo is a C++ linear algebra library; mlpack uses its matrix data type.
    arma::mat data(arr, 17, 4);
    
    /*
     * Load the data from a file. mlpack does not provide an example dataset, 
     * so I've included a tiny one.
     *
     * 'data' is a helper class in mlpack that facilitates saving and loading 
     * matrices and models.
     *
     * Pass the filename, matrix to hold the data, and set fatal = true to have
     * it throw an exception if there is an issue.
     *
     * 'Load' excepts comma-separated and tab-separated text files, and will 
     * infer the format.
     */
    //data::Load("data.csv", data, true);
    
    /* 
     * Create a NeighborSearch model. The parameters of the model are specified
     * with templates:
     *  - Sorting method: "NearestNeighborSort" - This class sorts by increasing
     *    distance.
     *  - Distance metric: "ManhattanDistance" - The L1 distance, sum of absolute
     *    distances.
     *
     * Pass the reference dataset (the vectors to be searched through) to the
     * constructor.
     */
     NeighborSearch<NearestNeighborSort, ManhattanDistance> nn(data);
    
    /*
     * Create the matrices to hold the results of the search. 
     *   neighbors [k  x  n] - Indeces of the nearest neighbor(s). 
     *                         One column per data query vector and one row per
     *                        'k' neighbors.
     *   distances [k  x  n] - Calculated distance values.
     *                         One column per data query vector and one row per
     *                        'k' neighbors.
     */
    arma::Mat<size_t> neighbors;
    arma::mat distances(distarr, 17, 17); 
    
    /*
     * Find the nearest neighbors. 
     *
     * If no query vectors are provided (as is the case here), then the 
     * reference vectors are searched against themselves.
     *
     * Specify the number of neighbors to find, k = 1, and provide matrices
     * to hold the result indeces and distances.
     */ 
    nn.Search(1, neighbors, distances);
    
    // Print out each neighbor and its distance.
    for (size_t i = 0; i < neighbors.n_elem; ++i)
    {
        std::cout << "Nearest neighbor of point " << i << " is point "
        << neighbors[i] << " and the distance is " << distances[i] << ".\n";
    }

    
}

We are compiling this with the command:

icpc -std=c++11 -I/home/userxx/Downloads/pyMIC/include -I/usr/include/python2.7 -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/include -fPIC -qopenmp  -g -shared -mmic -o knn.so knn.cpp

And the output with OFFLOAD_REPORT=3 follows:

[root@localhost knn_mlpack_example]# python2.7 knn.py 
PYMIC: debug level set to 7
PYMIC: tracing is disabled
PYMIC: found 1 device(s)
PYMIC: created stream 0x169f6f0 for device 0
PYMIC: searching for liboffload_array.so in 
PYMIC:     looking for liboffload_array.so in /usr/lib64/python2.7/site-packages/pymic
PYMIC: loading '/usr/lib64/python2.7/site-packages/pymic/liboffload_array.so' on device 0
[Offload] [HOST]          [State]           Initialize logical card 0 = physical card 0
[Offload] [MIC 0] [File]                    src/pymicimpl_misc.cc
[Offload] [MIC 0] [Line]                    186
[Offload] [MIC 0] [Tag]                     Tag 0
[Offload] [HOST]  [Tag 0] [State]           Start target
[Offload] [HOST]  [Tag 0] [State]           Setup target entry: __offload_entry_pymicimpl_misc_cc_186target_lo_cb16b0e23fb7bc6a3c1ad05f43fef8d0icc06321843140yPzMH
[Offload] [HOST]  [Tag 0] [Signal]          signal : none
[Offload] [HOST]  [Tag 0] [Signal]          waits  : none
[Offload] [HOST]  [Tag 0] [State]           Gather copyin data: base=0x16af790 length=80289
[Offload] [HOST]  [Tag 0] [State]           Create target buffer: size=82225 offset=1936
[Offload] [HOST]  [Tag 0] [State]           Host->target pointer data 80289
[Offload] [HOST]  [Tag 0] [State]           Host->target copyin data 24 
[Offload] [HOST]  [Tag 0] [State]           Execute task on target
[Offload] [HOST]  [Tag 0] [State]           Target->host pointer data 320
[Offload] [MIC 0] [Tag 0] [State]           Start target entry: __offload_entry_pymicimpl_misc_cc_186target_lo_cb16b0e23fb7bc6a3c1ad05f43fef8d0icc06321843140yPzMH
[Offload] [MIC 0] [Tag 0] [Var]             __offload_stack_ptr__ZN5pymic19target_load_libraryEiRKSsRSsRm.176  NOCOPY
[Offload] [MIC 0] [Tag 0] [Var]             size_in  IN
[Offload] [MIC 0] [Tag 0] [Var]             data  IN
[Offload] [MIC 0] [Tag 0] [Var]             bufsz  IN
[Offload] [MIC 0] [Tag 0] [Var]             __offload_stack_ptr__ZN5pymic19target_load_libraryEiRKSsRSsRm.176  OUT
[Offload] [MIC 0] [Tag 0] [Var]             __offload_stack_ptr__ZN5pymic19target_load_libraryEiRKSsRSsRm.176  OUT
[Offload] [MIC 0] [Tag 0] [Var]             handle_device_ptr  OUT
[Offload] [MIC 0] [Tag 0] [Var]             tempname_cstr_sz  INOUT
[Offload] [MIC 0] [Tag 0] [State]           Target->host copyout data   16
[Offload] [HOST]  [Tag 0] [CPU Time]        0.835535(seconds)
[Offload] [MIC 0] [Tag 0] [CPU->MIC Data]   80313 (bytes)
[Offload] [MIC 0] [Tag 0] [MIC Time]        0.001485(seconds)
[Offload] [MIC 0] [Tag 0] [MIC->CPU Data]   336 (bytes)

PYMIC: successfully loaded '/usr/lib64/python2.7/site-packages/pymic/liboffload_array.so' on device 0 with handle 0x7f168c0011c0
PYMIC: starting initialization of the offload infrastructure
PYMIC: loading supporting pyMIC libraries on all devices
PYMIC: searching for knn.so in 
PYMIC:     looking for knn.so in /usr/lib64/python2.7/site-packages/pymic
PYMIC:     looking for knn.so in 
PYMIC: loading 'knn.so' on device 0
[Offload] [MIC 0] [File]                    src/pymicimpl_misc.cc
[Offload] [MIC 0] [Line]                    186
[Offload] [MIC 0] [Tag]                     Tag 1
[Offload] [HOST]  [Tag 1] [State]           Start target
[Offload] [HOST]  [Tag 1] [State]           Setup target entry: __offload_entry_pymicimpl_misc_cc_186target_lo_cb16b0e23fb7bc6a3c1ad05f43fef8d0icc06321843140yPzMH
[Offload] [HOST]  [Tag 1] [Signal]          signal : none
[Offload] [HOST]  [Tag 1] [Signal]          waits  : none
[Offload] [HOST]  [Tag 1] [State]           Gather copyin data: base=0x16dc310 length=3100981
[Offload] [HOST]  [Tag 1] [State]           Create target buffer: size=3101765 offset=784
[Offload] [HOST]  [Tag 1] [State]           Host->target pointer data 3100981
[Offload] [HOST]  [Tag 1] [State]           Host->target copyin data 32 
[Offload] [HOST]  [Tag 1] [State]           Execute task on target
[Offload] [HOST]  [Tag 1] [State]           Target->host pointer data 320
[Offload] [MIC 0] [Tag 1] [State]           Start target entry: __offload_entry_pymicimpl_misc_cc_186target_lo_cb16b0e23fb7bc6a3c1ad05f43fef8d0icc06321843140yPzMH
[Offload] [MIC 0] [Tag 1] [Var]             __offload_stack_ptr__ZN5pymic19target_load_libraryEiRKSsRSsRm.176  NOCOPY
[Offload] [MIC 0] [Tag 1] [Var]             size_in  IN
[Offload] [MIC 0] [Tag 1] [Var]             data  IN
[Offload] [MIC 0] [Tag 1] [Var]             bufsz  IN
[Offload] [MIC 0] [Tag 1] [Var]             __offload_stack_ptr__ZN5pymic19target_load_libraryEiRKSsRSsRm.176  OUT
[Offload] [MIC 0] [Tag 1] [Var]             __offload_stack_ptr__ZN5pymic19target_load_libraryEiRKSsRSsRm.176  OUT
[Offload] [MIC 0] [Tag 1] [Var]             handle_device_ptr  OUT
[Offload] [MIC 0] [Tag 1] [Var]             tempname_cstr_sz  INOUT
[Offload] [MIC 0] [Tag 1] [State]           Target->host copyout data   16
[Offload] [HOST]  [Tag 1] [CPU Time]        0.020388(seconds)
[Offload] [MIC 0] [Tag 1] [CPU->MIC Data]   3101013 (bytes)
[Offload] [MIC 0] [Tag 1] [MIC Time]        0.014470(seconds)
[Offload] [MIC 0] [Tag 1] [MIC->CPU Data]   336 (bytes)

caught!
Traceback (most recent call last):
  File "knn.py", line 9, in <module>
    library = device.load_library(("knn.so",))
  File "/usr/lib64/python2.7/site-packages/pymic/_tracing.py", line 128, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib64/python2.7/site-packages/pymic/offload_device.py", line 168, in load_library
    return OffloadLibrary(libraries[0], device=self)
  File "/usr/lib64/python2.7/site-packages/pymic/offload_library.py", line 125, in __init__
    filename)
  File "src/pymic_libxstream.pyx", line 238, in pymic.pymic_libxstream.pymic_library_load
  File "src/pymic_libxstream.pyx", line 231, in pymic.pymic_libxstream._c_pymic_library_load
pymic.offload_error.OffloadError: Could not load library 'knn.so' on device 0
[Offload] [MIC 0] [File]                    src/pymicimpl_misc.cc
[Offload] [MIC 0] [Line]                    258
[Offload] [MIC 0] [Tag]                     Tag 2
[Offload] [HOST]  [Tag 2] [State]           Start target
[Offload] [HOST]  [Tag 2] [State]           Setup target entry: __offload_entry_pymicimpl_misc_cc_258target_un_0cc318b72500803c42a213bcdc3aa259icc06321843140yPzMH
[Offload] [HOST]  [Tag 2] [Signal]          signal : none
[Offload] [HOST]  [Tag 2] [Signal]          waits  : none
[Offload] [HOST]  [Tag 2] [State]           Gather copyin data: base=0x1667c78 length=22
[Offload] [HOST]  [Tag 2] [State]           Create target buffer: size=3214 offset=3192
[Offload] [HOST]  [Tag 2] [State]           Host->target pointer data 22
[Offload] [HOST]  [Tag 2] [State]           Host->target copyin data 24 
[Offload] [HOST]  [Tag 2] [State]           Execute task on target
[Offload] [HOST]  [Tag 2] [State]           Target->host pointer data 256
[Offload] [MIC 0] [Tag 2] [State]           Start target entry: __offload_entry_pymicimpl_misc_cc_258target_un_0cc318b72500803c42a213bcdc3aa259icc06321843140yPzMH
[Offload] [MIC 0] [Tag 2] [Var]             __offload_stack_ptr__ZN5pymic21target_unload_libraryEiRKSsm.30  NOCOPY
[Offload] [MIC 0] [Tag 2] [Var]             handle  IN
[Offload] [MIC 0] [Tag 2] [Var]             bufsz  IN
[Offload] [MIC 0] [Tag 2] [Var]             tempname_cstr  IN
[Offload] [MIC 0] [Tag 2] [Var]             __offload_stack_ptr__ZN5pymic21target_unload_libraryEiRKSsm.30  OUT
[Offload] [MIC 0] [Tag 2] [Var]             errorcode  OUT
[Offload] [MIC 0] [Tag 2] [State]           Target->host copyout data   4
[Offload] [HOST]  [Tag 2] [CPU Time]        0.001950(seconds)
[Offload] [MIC 0] [Tag 2] [CPU->MIC Data]   46 (bytes)
[Offload] [MIC 0] [Tag 2] [MIC Time]        0.000585(seconds)
[Offload] [MIC 0] [Tag 2] [MIC->CPU Data]   260 (bytes)

offload error: cannot unload library from the device 0 (error code 14)

Would appreciate any help.

Cypthon Offload Support

pyMIC requires programmers to write native kernels in C/C++ (or Fortran) to run on a coprocessor device. Support for Cython code would make programmers' lifes a bit easier, as it would allow them to write more Pythonesque code and then compile it to native coprocessor code.

A problem occured when reading .nii.gz

Dear authors,
thank you for sharing this helpfull project.
I have occured one problem when reading the DICOM CT volume (.nii.gz) and its labeled image(.nii.gz)。
In the train function of net_run_agent.py, when i run this sentence "labels_prob = self.convert_tensor_type(data['label_prob'])", the error "KeyError: 'label_prob'" appeared.
I have checked the loaded training data, there is a attribute named "label", but not "label_prob".
Could you please help me to solve this problem?
Is there some requirements for the input labeled images? Thanks!

Throw exception when exceeding supported no. of kernel args

The pyMIC code should intercept kernel invocations that exceed the number of kernel arguments that is supported by the libxstream implementation.

Using vtune with pymic.

Do you have an example of profiling code offloaded using pyMIC with vTune?