pthom/cvnp

cvnp: pybind11 casts between numpy and OpenCV, with shared memory

cvnp's Issues

what is dt.char_()

Did not exist char_() in pybind11::dtype
where is it?

Support non-contiguous arrays?

Is there a technical reason why this isn't supported or is it just you didn't want to implement it? I'd be interested in taking a shot if it's the latter, but wanted to make sure first that I knew what I was getting myself into first.

Python to c++ non-continuous image issue

Hello, I have been running into this issue, not sure if I am doing something wrong, or misunderstood the capabilities of your libraries.

on python side following is run:

img2 = cv2.imread('testImg.png',cv2.IMREAD_COLOR)
img3Test = img2[:,:,1]
detector2.showImage(img3Test)

On c++ side this is run

.def("showImage", &PYARobustChessboardDetector::showImage)

void showImage(cv::Mat mat) {
        cv::imshow("Mat", mat);
        cv::waitKey(0);
        cv::destroyAllWindows();
    }

and imshow shows this

if I enforce continuousness
res4 = detector2.processImage(np.ascontiguousarray(img3Test))
It is as expected

I assume your libraries are properly linked, as I would expect error be thrown when I try to load Mat from python, without support.

Though to be fair as I write this, you claim continuous data support, not contiguous so this issue may be outside of the scope of the project.

Split source files and header files

I'm currently playing with the idea of packing this up with conan for easy accessibility. At first attempt it requires workarounds for dealing with the header files.

Would you be open to a PR that splits the header files into an include directory so that it's easier to consume downstream?

shall the array copy when without share_memory?

when shared memory not used , it seems that
m is out of scope, it is thus m.data freed, and the returned array directly points to the old address on the stack!`:

cvnp/cvnp/cvnp.cpp

Line 91 in 602aa26

, m.data

pybind11::array make_array()
{
    cv::Mat m(cv::Size(10, 10), CV_8UC1);               // create a matrix on the stack
    pybind11::array a = cvnp::mat_to_nparray(m, false); 
    return a;                                                        
}  // Here be dragons, when closing the scope!
   // m is now out of scope, it is thus freed, 
   // and the returned array directly points to the old address on the stack!

Question: supporting non-continuous Mat ?

I tried copying some of your code to my project.

When I ran my Python program, I got the error

ValueError: Only continuous Mats supported.

from the method mat_to_nparray(const cv::Mat& m

where it checks if (!m.isContinuous())

What is needed to be modified to support non-continuous Mats ?

Reused data crashes

I'm going to work on a smaller test/reproducer for this, but leaving this here for now in case you have thoughts on how to address this. Basically, if there's a function that takes a mat argument, and returns the same mat, then the returned numpy array makes a capsule to keep the mat alive (which is great), but there's nothing to keep the original numpy array alive as far as I can tell.

This can be seen in cv::Mat nparray_to_mat(pybind11::array& a) where the mat is created with the a.mutable_data, but it's not trying to keep it alive. I believe we need some way to notice that the mat is holding a numpy array and return that original object.

I think the opencv python support works around this by having a custom allocator + umat.

• Allocator uses existing numpy memory: https://github.com/opencv/opencv/blob/4.x/modules/python/src2/cv2_numpy.cpp#L13
• Allocator makes new numpy memory: https://github.com/opencv/opencv/blob/4.x/modules/python/src2/cv2_numpy.cpp#L26
• Mat sets the custom allocator: https://github.com/opencv/opencv/blob/4.x/modules/python/src2/cv2_convert.cpp#L301

Reference: robotpy/mostrobotpy#76

Undefined symbol: _ZTIN2cv12MatAllocatorE

I am attempting to use cvnp to handle the type casting for numpy arrays and cv::Mat for a project. I have the following directory structure:

• project_dir
  • cpp (This handles all my source code and CMakeLists.txt for installing as a library)
  • python (This handles creating bindings for the cpp folder)
  • extern
    • cvnp
    • pybind11
  • CMakeLists.txt

When I build my project everything succeeds with the following warning:

/home/jcdavis/pteranodon/extern/pybind11/include/pybind11/cast.h:38:7: note: type ‘struct type_caster’ itself violates the C++ One Definition Rule
38 | class type_caster : public type_caster_base {};
| ^
/home/jcdavis/pteranodon/extern/cvnp/cvnp/cvnp.h:137:16: note: the incompatible type is defined here
137 | struct type_castercv::Mat
| ^
make[3]: Leaving directory '/home/jcdavis/pteranodon/build_ext'

Once I install my project and attempt to import in python I get the following error:

import pteranodon_ext
Traceback (most recent call last):
File "", line 1, in
ImportError: /home/jcdavis/.local/lib/python3.8/site-packages/pteranodon_ext.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZTIN2cv12MatAllocatorE

Here is my CMakeLists.txt for reference:

cmake_minimum_required(VERSION 3.4)
project(pteranodon_ext VERSION 0.1.0)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

if(NOT CMAKE_BUILD_TYPE)
  set(CMAKE_BUILD_TYPE Release)
endif()

set(CMAKE_CXX_FLAGS "-O3")
set(CMAKE_CXX_FLAGS_RELEASE "-O3")

include_directories(${CMAKE_CURRENT_SOURCE_DIR}/cpp/include/headers)
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/python)

add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/extern/pybind11)
add_subdirectory(${CMAKE_CURRENT_SOURCE_DIR}/extern/cvnp)

find_package(OpenCV REQUIRED)
include_directories(${OpenCV_INCLUDE_DIRS})

file (GLOB_RECURSE SOURCE_FILES CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/cpp/src/*.cpp)
file (GLOB_RECURSE HEADER_FILES CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/cpp/include/headers/*.hpp)
file (GLOB_RECURSE PYTHON_FILES CONFIGURE_DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/python/*.cpp ${CMAKE_CURRENT_SOURCE_DIR}/python/*.hpp)

pybind11_add_module(pteranodon_ext 
	${SOURCE_FILES}
	${HEADER_FILES}
	${PYTHON_FILES}
)

target_link_libraries(pteranodon_ext PUBLIC ${OPENCV_LIBS} PRIVATE cvnp pybind11::pybind11)

find_package(PythonInterp 3 REQUIRED)
find_package(PythonLibs 3 REQUIRED)

exec_program(${PYTHON_EXECUTABLE}
             ARGS "-c \"import sysconfig; print(sysconfig.get_paths()['purelib'])\""
             OUTPUT_VARIABLE PYTHON_LIBRARY_DIR
             RETURN_VALUE PYTHON_LIBRARY_DIR_NOT_FOUND
            )
if(PYTHON_LIBRARY_DIR_NOT_FOUND)
    message(FATAL_ERROR "Python library directory not found")
endif()

install(TARGETS pteranodon_ext
  COMPONENT python
  LIBRARY DESTINATION "${PYTHON_LIBRARY_DIR}"
)

My typical build process is as follows:

mkdir -p build && cd build && cmake -S ../ -B ./ && make
cd build && make install

I am wondering if there are any insights on this issues, thank you!