ssciwr/pybind11-numpy-example

Example of using pybind11 with numpy

pybind11-numpy-example

What

A simple example of how to use pybind11 with numpy.

This C++/Python library creates a std::vector of 16-bit ints, and provides a Python interface to the contents of this vector in a few different ways:

• a Python List (copy the data)
• a NumPy ndarray (copy the data).
• a NumPy ndarray (move the data).

Why

Python Lists are great! However, when storing many small elements of the same type, a Numpy array is much faster and uses a lot less memory:

How

The pybind11 code is in src/pybind11_numpy_example_python.cpp.

The python package is defined in pyproject.toml and uses scikit-build-core.

Each tagged commit triggers a GitHub action job which uses cibuildwheel to build and upload wheels to PyPI.

The scripts used to generate the above plots are in scripts.

This repo was quickly set up using the SSC C++ Project Cookiecutter.

pybind11-numpy-example's Issues

install TARGETS given target "pybind11-numpy-example" which does not exist in this directory

Hello, when I try to compile this example with python setup.py install, the following error occurs, does someone know how to fix this?

running install
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  setuptools.SetuptoolsDeprecationWarning,
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing pybind11_numpy_example.egg-info/PKG-INFO
writing dependency_links to pybind11_numpy_example.egg-info/dependency_links.txt
writing top-level names to pybind11_numpy_example.egg-info/top_level.txt
reading manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
-- pybind11 v2.10.0 
CMake Error at CMakeLists.txt:52 (install):
  install TARGETS given target "pybind11-numpy-example" which does not exist
  in this directory.


-- The following REQUIRED packages have been found:

 * PythonInterp (required version >= 3.6)
 * PythonLibsNew

-- Configuring incomplete, errors occurred!
See also "/data/code_zxt/pybind11-numpy-example/build/temp.linux-x86_64-cpython-37/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "setup.py", line 101, in <module>
    "License :: OSI Approved :: MIT License",
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
    self.run_command(cmd)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "setup.py", line 27, in run
    self.build_extension(ext)
  File "setup.py", line 62, in build_extension
    ["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/data/code_zxt/pybind11-numpy-example', '-DBUILD_DOCS=OFF', '-DBUILD_TESTING=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/code_zxt/pybind11-numpy-example/build/lib.linux-x86_64-cpython-37/', '-DPYTHON_EXECUTABLE=/home/zxt/anaconda3/envs/seg2/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
(seg2) zxt@octopus-A003:/data/code_zxt/pybind11-numpy-example$ clear
(seg2) zxt@octopus-A003:/data/code_zxt/pybind11-numpy-example$ python setup.py install
running install
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py:37: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  setuptools.SetuptoolsDeprecationWarning,
/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/easy_install.py:147: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  EasyInstallDeprecationWarning,
running bdist_egg
running egg_info
writing pybind11_numpy_example.egg-info/PKG-INFO
writing dependency_links to pybind11_numpy_example.egg-info/dependency_links.txt
writing top-level names to pybind11_numpy_example.egg-info/top_level.txt
reading manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
adding license file 'LICENSE.md'
writing manifest file 'pybind11_numpy_example.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
-- pybind11 v2.10.0 
CMake Error at CMakeLists.txt:52 (install):
  install TARGETS given target "pybind11-numpy-example" which does not exist
  in this directory.


-- The following REQUIRED packages have been found:

 * PythonInterp (required version >= 3.6)
 * PythonLibsNew

-- Configuring incomplete, errors occurred!
See also "/data/code_zxt/pybind11-numpy-example/build/temp.linux-x86_64-cpython-37/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "setup.py", line 101, in <module>
    "License :: OSI Approved :: MIT License",
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/__init__.py", line 87, in setup
    return distutils.core.setup(**attrs)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 185, in setup
    return run_commands(dist)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
    dist.run_commands()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
    self.run_command(cmd)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 74, in run
    self.do_egg_install()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install.py", line 123, in do_egg_install
    self.run_command('bdist_egg')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 165, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/bdist_egg.py", line 151, in call_command
    self.run_command(cmdname)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/command/install_lib.py", line 112, in build
    self.run_command('build_ext')
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
    self.distribution.run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/dist.py", line 1217, in run_command
    super().run_command(command)
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
    cmd_obj.run()
  File "setup.py", line 27, in run
    self.build_extension(ext)
  File "setup.py", line 62, in build_extension
    ["cmake", ext.sourcedir] + cmake_args, cwd=self.build_temp, env=env
  File "/home/zxt/anaconda3/envs/seg2/lib/python3.7/subprocess.py", line 363, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/data/code_zxt/pybind11-numpy-example', '-DBUILD_DOCS=OFF', '-DBUILD_TESTING=OFF', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/data/code_zxt/pybind11-numpy-example/build/lib.linux-x86_64-cpython-37/', '-DPYTHON_EXECUTABLE=/home/zxt/anaconda3/envs/seg2/bin/python', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.

conda-forge package

This repo currently serves as a simple minimal example of how to build & release binary wheels for a python / pybind11 project on PyPI.

It would be nice to also publish it on conda-forge to show how this works.

add ndarray example without copying data

see pybind/pybind11#1042 (comment)