- Read this in a nicer format on Medium.
- Follow my publication 'practical coding' on Medium for more articles like this.
- Tutorial on more advanced pybind11 features here.
This tutorial shows how to set up a pybind11 project with CMake for wrapping a C++ library into Python.
The final result will be:
- A
C++project you can build independent ofpybind11. - A
Pythonlibrary generated from wrapping theC++code. - Both using
CMake.
We will use the outer (current) working directory to build python, and an inner directory called cpp to build the C++ code. First make a C++ directory.
mkdir cpp
cd cpp
Next, we will initialize a C++ project. Two ways (of many more) are:
-
Using
VS Code. Install theCMake Toolsextension. Then, bring up the command pallette and selectCMake: Quick start. Follow the prompts and enter a name - I choseautomobile. When prompted for library or executable, chooselibrary. Your directory should now look like this:cpp/build/ cpp/automobile.cpp cpp/CMakeLists.txtWe will separate the source and header files - this is always good practice. In the
cppdirectory, make two new directories:cd cpp mkdir include mkdir srcand move the source file:
mv automobile.cpp src/In the include directory, we would like to have a single header to import. This way, we could later simply
#include <automobile>. We can organize it as follows:cd cpp/include mkdir automobile_bits touch automobileFinally, let us create a header file in the
cpp/include/automobile_bitsdirectory:cd cpp/include/automobile_bits touch motorcycle.hppThe final directory structure should now look like this:
cpp/build cpp/CMakeLists.txt cpp/include/automobile cpp/include/automobile_bits/motorcycle.hpp cpp/src/motorcycle.cpp -
Manually create the files and directories, such that the final structure is:
cpp/build cpp/CMakeLists.txt cpp/include/automobile cpp/include/automobile_bits/motorcycle.hpp cpp/src/motorcycle.cpp
We will need to edit the current CMakeLists.txt such that it can find the header and source files. I edited mine to read as follows:
cmake_minimum_required(VERSION 3.1)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
project(automobile VERSION 0.1.0)
# Include dir
include_directories(/usr/local/include)
# Src
AUX_SOURCE_DIRECTORY(src SRC_FILES)
# Headers
set(PROJECT_SOURCE_DIR "src")
set(PROJECT_INCLUDE_DIR "include/automobile_bits")
# Source files
set(SOURCE_FILES
${PROJECT_INCLUDE_DIR}/motorcycle.hpp
${PROJECT_SOURCE_DIR}/motorcycle.cpp
)
# Set up such that XCode organizes the files correctly
source_group(TREE ${CMAKE_CURRENT_SOURCE_DIR} FILES ${SOURCE_FILES})
# Add library
add_library(automobile SHARED ${SOURCE_FILES})
# Include directories
target_include_directories(automobile PRIVATE include/automobile_bits)
# Install
install(TARGETS automobile DESTINATION lib)
# Install the headers
install(FILES include/automobile DESTINATION include)
# Create base directory
install(DIRECTORY include/automobile_bits DESTINATION include)
Let's also give the motorcycle.hpp and motorcycle.cpp files some reasonable content. For the header:
#include <string>
#ifndef CAR_H
#define CAR_H
namespace vehicles {
class Motorcycle {
private:
/// Name
std::string _name;
public:
/// Constructor
Motorcycle(std::string name);
/// Get motorcycle name
/// @return Motorcycle name
std::string get_name() const;
/// Drive the motorcycle
void ride() const;
};
}
#endif
and the source:
#include "../include/automobile_bits/motorcycle.hpp"
#include <iostream>
namespace vehicles {
Motorcycle::Motorcycle(std::string name) {
_name = name;
}
std::string Motorcycle::get_name() const {
return _name;
}
void Motorcycle::ride() const {
std::cout << "Zoom Zoom" << std::endl;
}
}
Yes! I know they're dumb. Note that we introduced a namespace vehicles - this is always a good idea.
We also need to have the header file find the actual library. Edit the include/automobile file to read:
#ifndef AUTOMOBILE_LIBRARY_H
#define AUTOMOBILE_LIBRARY_H
#include "automobile_bits/motorcycle.hpp"
#endif
We can now already build the library:
-
Using the command line:
cd cpp/build cmake .. make make install -
Using your favorite IDE, e.g.
XCode:cd cpp/build cmake .. -GXcodeshould generate
automobile.xcodeprojectin thebuilddirectory.
Either way, you should get the library to build and install.
Before we go on to wrapping the library into Python, let's create a test for the C++ library (not a real test, just somewhere for us to mess around!).
Create a new directory in cpp:
cd cpp
mkdir tests
Here we will again set up a CMake project for our test. Make the directory structure look as follows:
cpp/tests/CMakeLists.txt
cpp/tests/src/test_cpp.cpp
Edit the test_cpp.cpp file to read:
#include <automobile>
#include <iostream>
int main() {
vehicles::Motorcycle c("Yamaha");
std::cout << "Made a motorcycle called: " << c.get_name() << std::endl;
c.ride();
return 0;
}
and edit the CMakeLists.txt file:
cmake_minimum_required(VERSION 3.1)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
project(test_cpp)
include_directories(/usr/local/include)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY ${CMAKE_BINARY_DIR}/../bin)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_RELEASE ${CMAKE_BINARY_DIR}/../bin)
set(CMAKE_RUNTIME_OUTPUT_DIRECTORY_DEBUG ${CMAKE_BINARY_DIR}/../bin)
find_library(AUTOMOBILE_LIB automobile HINTS /usr/local/lib/)
add_executable(test_cpp src/test_cpp.cpp)
target_link_libraries(test_cpp PUBLIC ${AUTOMOBILE_LIB})
Make and run that bad boy using XCode as before, or from the command line:
mkdir build
cd build
cmake ..
make
cd ../bin
./test_cpp
Note that the binary will be in the bin directory. The output should be:
Made a motorcycle called: Yamaha
Zoom Zoom on road: mullholland
Finally, let's get to wrapping the library into a Python. We're moving up a directory! In the main directory, let's make a new directory called python. It will hold all the glue code:
mkdir python
We also need a CMakeLists.txt file, with contents:
cmake_minimum_required(VERSION 3.1)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)
if(NOT CMAKE_BUILD_TYPE)
set(CMAKE_BUILD_TYPE Release)
endif()
set(CMAKE_CXX_FLAGS "-O3")
set(CMAKE_CXX_FLAGS_RELEASE "-O3")
project(automobile)
include_directories("${CMAKE_SOURCE_DIR}/cpp/include/automobile_bits")
include_directories("${CMAKE_SOURCE_DIR}/python")
file (GLOB SOURCE_FILES "cpp/src/*.cpp")
file (GLOB HEADER_FILES "cpp/include/automobile_bits/*.hpp")
file (GLOB PYTHON_FILES "python/*.cpp" "python/*.hpp")
# Set up such that XCode organizes the files
source_group(TREE ${CMAKE_CURRENT_SOURCE_DIR} FILES ${SOURCE_FILES} ${HEADER_FILES} ${PYTHON_FILES} )
find_package(pybind11 REQUIRED)
If pybind11 hasn't been installed in the system (e.g.: conda install -c conda-forge pybind11), you can use the pybind11.cmake file to fetch the package (see CMakeLists.txt as an example).
pybind11_add_module(automobile
${SOURCE_FILES}
${HEADER_FILES}
${PYTHON_FILES}
)
target_link_libraries(automobile PUBLIC)
install(TARGETS automobile
COMPONENT python
LIBRARY DESTINATION "${PYTHON_LIBRARY_DIR}"
)
You should be ready to build your Python library! Try:
mkdir build
cd build
cmake .. -DPYTHON_LIBRARY_DIR="/path/to/site-packages" -DPYTHON_EXECUTABLE="/path/to/executable/python3"
make
make install
As usual, you could also generate code using a generator for your favorite IDE, e.g. by adding -GXcode to the cmake command. My paths were:
DPYTHON_LIBRARY_DIR="/Users/USERNAME/opt/anaconda3/lib/python3.7/site-packages"
DPYTHON_EXECUTABLE="/Users/USERNAME/opt/anaconda3/bin/python3"
Note that if you are lazy like me, you can try to add for testing:
set(PYTHON_LIBRARY_DIR "/Users/USERNAME/opt/anaconda3/lib/python3.7/site-packages")
set(PYTHON_EXECUTABLE "/Users/USERNAME/opt/anaconda3/bin/python3")
in your CMakeLists.txt - obviously not a good trick for production!
Fire up python (make sure it's the same as you specified in PYTHON_EXECUTABLE above) and try:
>>> import automobile
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: dynamic module does not define module export function (PyInit_automobile)
You got a nice fat error, but that's OK! We didn't write the glue code yet, but at least your CMake is working and Python can find your library.
Now for the actual logic of wrapping the C++ code into Python. It will take place in the python directory. First create a file which will define the "module export function" that python was complaning about in the last part:
touch python/automobile.cpp
Give it the following content:
#include <pybind11/pybind11.h>
namespace py = pybind11;
void init_motorcycle(py::module &);
namespace mcl {
PYBIND11_MODULE(automobile, m) {
// Optional docstring
m.doc() = "Automobile library";
init_motorcycle(m);
}
}
Next, we will define the init_motorcycle method that was declared. We will do this in a separate file:
touch python/motorcycle.cpp
Edit it to read:
#include "../cpp/include/automobile_bits/motorcycle.hpp"
#include <pybind11/stl.h>
#include <pybind11/pybind11.h>
namespace py = pybind11;
void init_motorcycle(py::module &m) {
py::class_<vehicles::Motorcycle>(m, "Motorcycle")
.def(py::init<std::string>(), py::arg("name"))
.def("get_name",
py::overload_cast<>( &vehicles::Motorcycle::get_name, py::const_))
.def("ride",
py::overload_cast<std::string>( &vehicles::Motorcycle::ride, py::const_),
py::arg("road"));
}
I always find the code itself to be the best explanation, but some pointers:
py::class_<vehicles::Motorcycle>(m, "Motorcycle")defines the class. The"Motorcycle"defines the name of the class inPython- you could change it if you want! Notice also the appearance of the namespace..def(py::init<std::string>(), py::arg("name"))defines the constructor. Thepy::arg("name")allows you to use named arguments inPython..def("get_name", py::overload_cast<>( &vehicles::Motorcycle::get_name, py::const_))wraps theget_namemethod. Note how theconstdeclaration is wrapped..def("ride", py::overload_cast<std::string>( &vehicles::Motorcycle::ride, py::const_), py::arg("road"));wraps theridemethod. The arguments to the method are declared inpy::overload_cast<std::string>(separated by commas if multiple), and can again be named usingpy::arg("road"). Also note the semicolon at the end - often forgotten, but this should be properC++code.
You can now test your library. Run make and make install again to rebuild and install the library.
Fire up python and try it:
import automobile
c = automobile.Motorcycle("Yamaha")
print("Made a motorcycle called: %s" % c.get_name())
c.ride("mullholland")
should give the same output as before:
Made a motorcycle called: Yamaha
Zoom Zoom on road: mullholland
You could make another test script with those contents, located in a directory tests/test.py.
That's it for this tutorial. The nice part about this setup is that you can build your C++ project in peace from the cpp directory, and then at the end in the outer layer worry about wrapping it into Python.
You can read about more advanced pybindy11 features in another tutorial here.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent