naibaf7/pygreentea

pygreentea's Issues

citation for publications using this code?

is this the right document to cite if using this code in published data?

setup.py vulnerable to shell command injection

Script setup.py is vulnerable to shell command injection since it uses os.system(). This function and os.popen() are deprecated in Python nowadays. It is recommended to use the subprocess module instead. The first argument path in clone_malis() and clone_caffe() would be vulnerable to shell command injection (see github.com/python-pillow/Pillow/issues/1520) if the string argument would terminate the git clone command early and add additional commands (e.g. os.system('git clone path;xterm')). This would first execute git clone and then execute xterm. It is therefore recommended to change:

os.system('git clone %s' % path)

to

subprocess.call(['git', 'clone', '', path])

Can't find numpy/arrayobject.h

Hello,

In addition to the "usual" two bugs with libhdf5, I hit another irritating roadblock.

So, after installing dependencies, and setting caffe to auto-download in the config.py script (maybe this should be the default?), I eventually hit an error about not finding numpy/arraybuffer.h:

==== PYGT: Compiling Caffe/Greentea ============================================
make: Nothing to be done for 'all'.
CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
python/caffe/_caffe.cpp:12:36: fatal error: shim/numpy/arrayobject.h: No such file or directory
compilation terminated.
Makefile:627: recipe for target 'python/caffe/_caffe.so' failed
make: *** [python/caffe/_caffe.so] Error 1

I've tried a few things to fix this, hackishly, like making symlinks to what I think it's looking for, and changing _caffe.cpp so it points to the header file as an absolute path, but the C preprocessor isn't my forté and I can't get past this error. :/

My numpy includes:

In [1]: import numpy
In [2]: numpy.get_include()
Out[2]: '/usr/local/lib/python3.5/dist-packages/numpy/core/include'

Would appreciate help getting this alive, I'd love to start using Caffe on OpenCL but the installation instructions for the opencl branch on caffe are..minimal, and this glue script doesn't seem to work correctly in its current state on Ubuntu 16.04.

Also, the caffe makefile.config appears to point to Python2.7, where it should probably be using some modern version of Python. Changing this didn't affect the above bug.

Thanks!

Installation/Training/Inference Instructions

@naibaf7 Can you please be kind enough to provide some instructions for installation/training and inference?

1. I have "already" caffe compiled and installed (opencl with Libdnn, ubuntu).
   Now do I set the option compile_caffe = false in config.py? How will the pygreen setup find the installed caffe lib? or Just run setup.py?

2. The net.proto for example will still need INTEL_SPATIAL get opencl acceleration? I dont see them here
   https://github.com/naibaf7/PyGreentea/blob/master/examples/2D_usk_malis_softmax/net.prototxt

3. Any idea how many images you used to get good accuracy? I read that image medical datasets were limited and you use other means to increase the quantity of images?

4. For inference is test.py is the right place?

More questions once I starting running them. Thanks for the patience. :-)

PyCaffe has no attribute 'SolverParameter'

the train.py scripts in the examples both evoke the following error:

Traceback (most recent call last):
  File "./train.py", line 56, in <module>
    solver_config = pygt.caffe.SolverParameter()
AttributeError: 'module' object has no attribute 'SolverParameter'

PyCaffe seems to be importing correctly because at that same point in the code, pygt.caffe has other PyCaffe objects (e.g. SGDSolver, get_solver, etc.).

i'm currently running Caffe 1.0.0-rc3. is there an different version of Caffe i should try running?

install problem

Already up-to-date.
==== PYGT: Compiling Caffe/Greentea ============================================
chmod +x src/caffe/greentea/cl_kernels.sh
src/caffe/greentea/cl_kernels.sh
CXX src/caffe/solvers/adam_solver.cpp
PROTOC src/caffe/proto/caffe.proto
CXX src/caffe/solver_factory.cpp
CXX src/caffe/solvers/nesterov_solver.cpp
CXX src/caffe/solvers/rmsprop_solver.cpp
CXX src/caffe/solvers/adagrad_solver.cpp
CXX src/caffe/solvers/sgd_solver.cpp
CXX src/caffe/solvers/adadelta_solver.cpp
CXX src/caffe/parallel.cpp
CXX src/caffe/data_transformer.cpp
CXX src/caffe/layer.cpp
CXX src/caffe/opencl/ocl_dev_ptr.cpp
CXX src/caffe/device.cpp
CXX src/caffe/internal_thread.cpp
CXX src/caffe/util/math_functions.cpp
CXX src/caffe/util/cl_fft_state.cpp
CXX src/caffe/util/db_lmdb.cpp
CXX src/caffe/util/im2col.cpp
In file included from src/caffe/device.cpp:17:0:
./include/caffe/greentea/cl_kernels.hpp:4:0: error: unterminated #ifdef
#ifdef USE_GREENTEA
^
make: *** [.build_release/src/caffe/device.o] error 1

//cl_kernels.hpp
// AUTOMATICALLY GENERATED FILE, DO NOT EDIT
#include
#include "caffe/common.hpp"
#ifdef USE_GREENTEA
#ifndef GREENTEA_CL_KERNELS_HPP_
#define GREENTEA_CL_KERNELS_HPP_
#include "caffe/greentea/greentea.hpp"
#include "viennacl/backend/opencl.hpp"
#include "viennacl/ocl/backend.hpp"
#include "viennacl/ocl/context.hpp"
#include "viennacl/ocl/device.hpp"
#include "viennacl/ocl/platform.hpp"
namespace caffe {
viennacl::ocl::program & RegisterKernels(viennacl::ocl::context *ctx);
viennacl::ocl::program & submit_conv_spatial_program(
viennacl::ocl::context *ctx, string name, string options);
std::string getKernelBundleName(int index);
int getKernelBundleCount();
template
std::string getKernelBundleSource(int index);
} // namespace caffe
#endif
#endif

I've checked the 'cl_kernels.hpp' but there is no lack of '#endif' in this file as above.
Any idea about this error please?Thx!