pybind/pybind11

Seamless operability between C++11 and Python

pybind11's Issues

Comparaison between enum type returns False

I have a C++ function which returns a enumerated type.
However, when I compare the returned value with another value the results is always false.

This happend in example4.cpp if you had:

EMyEnumeration returnEnum(){
  return EFirstEntry;
}
...
m.def("returnEnum", &returnEnum);

The following python code returns False each time:

import example
example.returnEnum() == example.EFirstEntry
example.returnEnum() == example.ESecondEntry

I think, the "eq" method is not defined and python compares address of the objects instead of values.

c++ function that modifies reference not reflected in python

Hi Jakob,
In C++ I have a class Space with public member vector<PointParticle> p and a non-class function template,

template<class Tspace>
   void cm2origo(Tspace::Tparticlevector &p) { /* modify p */ }

Wrapping is done with,

typedef Space<PointParticle> Tspace;

m.def("cm2origo",  &cm2origo<Tspace>);

py::class_<Tspace>(m, "Space")
  .def_readwrite("p", &Tspace::p)
  .def ... 

In python I can read/write to p as one would expect. However, when calling the exposed cm2origo function, changes to p are not transferred back to python. Any suggestions how to sort this out?

return_value_policy::reference fails if operator new is not public

In cases where operator new for a class is inaccessible, bound functions that return that type with the return_value_policy::reference will not compile

#include <memory>
#include <iostream>
#include <pybind11/pybind11.h>
using namespace pybind11;

class Foo {
    void *operator new(size_t bytes) throw();
};

Foo gFoo;
Foo& getstmt() {
    return gFoo;
}

PYBIND11_PLUGIN(example) {
    module m("example");
    class_<Foo>(m, "Foo");
    m.def("getstmt", getstmt, return_value_policy::reference);
    return m.ptr();
}

generating errors in type_caster_generic::cast

/usr/local/include/pybind11/cast.h:214:16: error: 'operator new' is a private member of 'Foo'
        return new type(*((const type *) arg));
               ^~~
/usr/local/include/pybind11/cast.h:202:80: note: in instantiation of function template specialization
      'pybind11::detail::type_caster<Foo, void>::copy_constructor<Foo, 0>' requested here
        return type_caster_generic::cast(&src, policy, parent, &typeid(type), &copy_constructor);
                                                                               ^
/usr/local/include/pybind11/pybind11.h:66:39: note: in instantiation of member function
      'pybind11::detail::type_caster<Foo, void>::cast' requested here
            handle result = cast_out::cast(
                                      ^
/usr/local/include/pybind11/pybind11.h:469:22: note: in instantiation of function template
      specialization 'pybind11::cpp_function::cpp_function<Foo &, pybind11::name, pybind11::sibling,
      pybind11::return_value_policy>' requested here
        cpp_function func(std::forward<Func>(f), name(name_),
                     ^
main.cpp:21:7: note: in instantiation of function template specialization 'pybind11::module::def<Foo
      &(&)(), pybind11::return_value_policy>' requested here
    m.def("getstmt", getstmt, return_value_policy::reference);
      ^
main.cpp:8:11: note: implicitly declared private here
    void *operator new(size_t bytes) throw();

The equivalent boost::python example does compile.

#include <memory>
#include <boost/python.hpp>

using namespace boost;
using namespace boost::python;

class Foo {
    void *operator new(size_t bytes) throw();
};

Foo gFoo;
Foo& getstmt() {
    return gFoo;
}

BOOST_PYTHON_MODULE(example)
{
    class_<Foo>("Foo");
    def("getstmt", getstmt, return_value_policy<reference_existing_object>());
}

option to see generated code?

Can we add an option that shows you what code it actually put to compile against?

pybind11 equivalent of boost with_custodian_and_ward

I am trying to use pybind11 instead of boost python and I didn't found the way may to set the correct call policies for my function call. (Maybe I missed something in the doc)

I have an object which takes in its constructor, a list as argument and the lifetime of the list and the object need to be tied. In boost I need to use with_custodian_and_ward call policy.

What is the equivalent in pybind11 ?

PYBIND11_NOINLINE inline

While trying to add icc support I stumbled upon several lines defining methods such as

PYBIND11_NOINLINE inline [...]

which makes probably no sense.

Is that intentionally?
Probably two individual PYBIND11_NOINLINE and PYBIND11_INLINE macros would be more useful (using always only once of them when trying to force to (prevent) inlining).

Add import

I'm not sure if I just can't find it, but it would be great to have the analog of boost::python::import to import a module as a pybind::object, and then to get its functions via attr.

Problems with integer conversions (again)

I have a new set of problems with integer conversions. It's not clear to me what's going on (and it requires another library to reproduce), but hopefully it will make sense to you.

You need mpi4py for this example.

Define a function through pybind11:

m.def("print_comm", [](int x) { std::cout << "Got comm: " << x << std::endl; });

Now test it from Python:

from mpi4py import MPI
w = MPI.COMM_WORLD
a = MPI._addressof(w)
print type(a)
# prints: <type 'int'>

print_comm(a)

The last command produces:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-16f94faa6020> in <module>()
----> 1 h.print_comm(a)

TypeError: Incompatible function arguments. The following argument types are supported:
    1. (int) -> None

I'm not sure how to debug this. If you have advice or any idea what's going on, I'd appreciate it.

Conversion across modules

The following works on Mac OS X, but not on Linux. (The obvious bits below, declarations of m and such, are ommited.)

// common.h

struct A {};

// a.cpp

#include "common.h"

PYBIND11_PLUGIN(a)
{
  py::class_<A>(m, "A")
      .def(py::init<>());
}

// b.cpp

#include "common.h"

PYBIND11_PLUGIN(b)
{
  m.def("test", [](A a) { fmt::print("test\n"); });
}

# test.py 

import a
import b

x = a.A()
b.test(x)

On Mac, this works fine. On Linux, I get:

TypeError: Incompatible function arguments. The following argument types are supported:
    1. (A) -> None

Can I use pybind11 from within a C++ application to create python plugins?

Hi,

I am looking for a way to create python plugins for my c++ applications. That is, load a user-defined .py file and run a python function in it with arguments from my running C++ application.

Can I do that with pybind11?

Inheritance with non-exposed base class

I came across this project, yesterday, and I'd like to say, I really like the project, improving on Boost.Python's environment and API quirks, and I'd like to use it in my C++-to-Python wrapper.

But as I was looking, yesterday, I came across a problem. I was trying to expose a leaf class of a large hierarchy, and encountered a problem when calling a method of the base class that wasn't overridden in the exposed class.

Here is the stripped down example of what I was trying to do:

#include <iostream>

class A
{
public:
    A() : x(1) {}
    virtual void f() { std::cout << "A::f" << std::endl; }
    virtual void g() { std::cout << "A::g" << std::endl; }
            void h() { std::cout << "A::h" << std::endl; }
            void i() { std::cout << "A::i" << std::endl; }

    int x;
};

class B : public A
{
public:
    virtual void f() { std::cout << "B::f" << std::endl; }
            void h() { std::cout << "B::h" << std::endl; }
};

#include "pybind11/pybind11.h"

namespace py = pybind11;

PYBIND11_PLUGIN(test)
{
    py::module m("test");

//  py::class_<A> a(m, "A");

    py::class_<B>(m, "B"/*, a*/)
        .def(py::init<>())
        .def("f", &B::f)
        .def("g", &B::g)
        .def("h", &B::h)
        .def("i", &B::i)
        .def_readwrite("x", &B::x)
    ;
}

If I then import the resulting library in Python, and try to use it, I get the following:

>>> import test
>>> b = test.B()
>>> b.i()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Incompatible function arguments. The following argument types are supported:
    1. (A) -> None

The same goes for calling 'g' and accessing 'x'.

I tried the same with Boost.Python, and it does not seem to have this problem. (Which is a pitty, 'cause again, I really like this slim header-only library, instead of Boost ;-) )

EDIT: Oh, yes, almost forgot. If you remove the 2 comments, and expose the A class and the fact that is is B's base class, everything works as it should.

numpy.h indirectly defines `I` - breaks following includes

The header pybind11/numpy.h includes complex.h which, at least in glibc, defines I as a preprocessor macro to denote the imaginary unit. Unfortunately I is a popular name for template parameters in e.g. the boost library. Therefore, the following two lines will fail to compile

#include <pybind11/numpy.h>
#include <boost/variant.hpp>

while the following three lines will work fine.

#include <pybind11/numpy.h>
#undef I
#include <boost/variant.hpp>

The compiler error is very long, however it starts with

include/boost/move/iterator.hpp:151:11: error: declaration of anonymous class must be a definition
template <class I>

which indicates that the I is the problem.

It took me some time to find the cause for that particular error. Do you think it would be reasonable to #undef I within pybind11/numpy.h to spare others from the same problem?

Add py::array::size() function?

Currently if I want to access a parameter such as array length, the best method to my knowledge is

size_t size = target.request().count;

There are two problems with this. First, the corresponding python name is size, so renaming count -> size would be nice.

Further, wouldnt It be nice to have access to this as a free function?

size_t size = target.size()

Accessing underlying C++ object from generic python object

I expect this is a trivial question, and apologies for that, but I cannot work out how to do this.

If I have exposed a C++ type to python using pybind11 and created several instances of it in python, stored in a list, what is the best way to obtain these instances in a different C++ function (jn this case the constructor of another class). I tried passing the list directly to a function expecting a std::vector<MyType> but this does not work. i.e.

py::class_<AnotherType>(m, "AnotherType")
    .def("__init__",
             [](AnotherType &instance, std::vector<MyType> instances) {
    ...
    }

I also wrote a lambda function in C++ to accept a py::list object; that worked, but I do not know how to extract the C++ types from the py::objects inside the list.

py::class_<AnotherType>(m, "AnotherType")
     .def("__init__",
             [](AnotherType &instance, py::list instances) {
                 for (auto inst : instances) {
                    //what do I do here?
                 }
    ...
    }

What I actually want in the end is a std::vector<std::shared_ptr<MyType>>.

What is the best way to achieve this?

builtin set type not supported

list and dict are suported by pybind11 (just like by boost::python), but support for the set builtin type is missing (also from boost::python -- see http://thread.gmane.org/gmane.comp.python.c++/15129). Could set be supported by pybind11?

Unary Operators Compilation Failure

I'm getting a compilation error when using a unary operator. Here's some sample code that exhibits the problem.

This is on Lubuntu 15.04 with gcc 4.9.2. Also on Kubuntu 14.04 with gcc 4.8.4.

#include <pybind11/pybind11.h>
#include <pybind11/operators.h>

namespace py = pybind11;

class foo
{
  public:
    foo(int f_) : m_f(f_)
    {
    }

    foo operator~() const
    {
      return foo(~m_f);
    }

  private:
    int m_f;
};

PYBIND11_PLUGIN(foo)
{
  py::module m("foom", "foom example");

  py::class_<foo>(m, "foo")
    .def(py::init<int>())

    .def(~py::self);

  return m.ptr();
}

I get the following errors:

In file included from test.cpp:2:0:
./pybind11/operators.h: In instantiation of ‘struct pybind11::detail::op_impl<(pybind11::detail::op_id)15, (pybind11::detail::op_type)2, foo, foo, pybind11::detail::undefined_t>’:
./pybind11/operators.h:52:29: required from ‘void pybind11::detail::op_<id, ot, L, R>::execute(pybind11::class_<Base, Holder>&, Extra&& ...) const [with Base = foo; Holder = std::unique_ptr<foo, std::default_delete >; Extra = {}; pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t]’
./pybind11/pybind11.h:703:9: required from ‘pybind11::class_<type, holder_type>& pybind11::class_<type, holder_type>::def(const pybind11::detail::op_<id, ot, L, R>&, Extra&& ...) [with pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t; Extra = {}; type = foo; holder_type = std::unique_ptr<foo, std::default_delete >]’
test.cpp:29:19: required from here
./pybind11/operators.h:96:17: error: ‘l’ was not declared in this scope
static auto execute(const L &l) -> decltype(expr) { return expr; }
^
./pybind11/operators.h:137:1: note: in expansion of macro ‘PYBIND11_UNARY_OPERATOR’
PYBIND11_UNARY_OPERATOR(invert, operator~, ~l)
^
./pybind11/operators.h: In instantiation of ‘void pybind11::detail::op_<id, ot, L, R>::execute(pybind11::class_<Base, Holder>&, Extra&& ...) const [with Base = foo; Holder = std::unique_ptr<foo, std::default_delete >; Extra = {}; pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t]’:
./pybind11/pybind11.h:703:9: required from ‘pybind11::class_<type, holder_type>& pybind11::class_<type, holder_type>::def(const pybind11::detail::op_<id, ot, L, R>&, Extra&& ...) [with pybind11::detail::op_id id = (pybind11::detail::op_id)15; pybind11::detail::op_type ot = (pybind11::detail::op_type)2; L = pybind11::detail::self_t; R = pybind11::detail::undefined_t; Extra = {}; type = foo; holder_type = std::unique_ptr<foo, std::default_delete >]’
test.cpp:29:19: required from here
./pybind11/operators.h:52:32: error: ‘execute’ is not a member of ‘op {aka pybind11::detail::op_impl<(pybind11::detail::op_id)15, (pybind11::detail::op_type)2, foo, foo, pybind11::detail::undefined_t>}’
class_.def(op::name(), &op::execute, std::forward(extra)...);
^

PYBIND11_DECLARE_HOLDER_TYPE small fix

This is just a small one:

template class type_caster<holder_type>

should be

template <> class type_caster<holder_type>

Tested with VC2015

Compilation error test case - arch linux

I clone master of pybind11

cmake .
make -j8

I got errors
http://pastebin.com/TrXpEwqL

gcc version 5.3.0 (GCC)
python 3.5m

What information do you need?
Thank

Doesn't build with Clang on Linux

-flto option prevents the example code from building on Linux with Clang (3.7). The problem is with the linker, it simply complains:

CMakeFiles/example.dir/example/example.cpp.o: file not recognized: File format not recognized

Removing -flto fixes everything. Apparently, this is not an unusual issue with Clang.

Passing both python functions and std::function as arguments interchangeably

I noticed that including pybind11/functional.h has some (probably) unintended side-effects.

If I include pybind11/functional.h I cannot pass std::function instances which I get from C++ code back to C++ functions. On the other hand without pybind11/functional.h, I cannot pass Python methods to C++ functions expecting std::function.

Is it possible to get both working, or are they mutually exclusive? I'm attaching a short example

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>  // comment/uncomment this

struct Functor {
    bool operator() (int a, int b) const {
      return a < b;
    }
    std::function<bool(int, int)> cast() {
      return std::function<bool(int, int)>(*this);
    }
};

struct Caller {
    bool call(std::function<bool(int, int)> func, int a, int b) {
      return func(a, b);
    }
};

namespace py = pybind11;

PYBIND11_PLUGIN(tmp) {
    py::module m("tmp", "pybind11 tmp plugin");

    py::class_<Caller>(m, "Caller").def(py::init<>())
                                   .def("call", &Caller::call);

    py::class_<std::function<bool(int, int)>>(m, "FuncType").def(py::init<>());

    py::class_<Functor>(m, "Functor").def(py::init<>())
                                     .def("__call__", [](const Functor& f, int a, int b) { return f(a, b); })
                                     .def("cast", &Functor::cast);
    return m.ptr();
}

import tmp

cppfunc = tmp.Functor()
caller = tmp.Caller()

def pyfunc(a, b):
    return a < b

try:
    print 'Calling C++ functor {0!r}'.format(cppfunc),
    res = cppfunc(1, 2)
    print 'OK'
except Exception as error:
    print 'FAILED with error {1!r}'.format(cppfunc, error)

try:
    print 'Calling C++ with Python function {0!r}'.format(pyfunc),
    caller.call(pyfunc, 1, 2)
    print 'OK'
except Exception as error:
    print 'FAILED with error {1!r}'.format(cppfunc, error)

stdfunc = cppfunc.cast()
try:
    print 'Calling C++ with C++ function {0!r}'.format(stdfunc),
    caller.call(stdfunc, 1, 2)
    print 'OK'
except Exception as error:
    print 'FAILED with error {1!r}'.format(cppfunc, error)

Without pybind11/functional.h

Calling C++ functor <tmp.Functor object> OK
Calling C++ with Python function <function pyfunc> FAILED with error TypeError('Incompatible function arguments. The following argument types are supported:\n    1. (Caller, std::function<bool (int, int)>, int32_t, int32_t) -> bool\n',)
Calling C++ with C++ function <tmp.FuncType object> OK

With pybind11/functional.h

Calling C++ functor <tmp.Functor object> OK
Calling C++ with Python function <function pyfunc> OK
Calling C++ with C++ function <built-in method  of PyCapsule object> FAILED with error TypeError('Incompatible function arguments. The following argument types are supported:\n    1. (Caller, function<(int32_t, int32_t) -> bool>, int32_t, int32_t) -> bool\n',)

DOCS: RAII style `NoGIL` class

This would be pretty easy to add, but would be quite nice to have by default (if not already present). Adding an RAII style object that releases the GIL when constructed and acquires the GIL when destructed. Having this object would make it easy to have blocks of C++ code that can run in parallel to the Python interpreter.

Efficiency question

I just have this maybe sort of silly question. For what I understand this library can help us create python wrapper of c++ code easily, right? So the compiled module for python, converted from c++, will be really efficient, since the functioning part of the code is actually written by c++. Is my understanding correct? Thanks.

Tests fail on OSX

I'm quite new to C programming in general so I might be doing something wrong:

I downloaded the zip, and unzipped it and cded into the resulting dictionary.

Run the following commands:
cmake .
make -j 4
make test

The first two completed what appears to be successfully (no mention of errors). But make test says 12 out of 12 test failed and gives the following error:
Errors while running CTest
make: *** [test] Error 8

I'm running OS X 10.11.1 and $c++ --version gives:
Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin15.0.0
Thread model: posix

Additionally trying the example from the docs gives a massive error:

$c++ -O3 -shared -std=c++11 -I ~/Desktop/pybind11-master/include `python-config --cflags --libs` example.cpp -o example.so
In file included from example.cpp:1:
In file included from /Users/mik/Desktop/pybind11-master/include/pybind11/pybind11.h:26:
In file included from /Users/mik/Desktop/pybind11-master/include/pybind11/cast.h:13:
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:294:14: error: call to constructor of 'pybind11::int_' is ambiguous
        int_ start(start_), stop(stop_), step(step_);
             ^     ~~~~~~
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:273:5: note: candidate constructor
    int_(int value) : object(PyLong_FromLong((long) value), false) { }
    ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:274:5: note: candidate constructor
    int_(size_t value) : object(PyLong_FromSize_t(value), false) { }
    ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit move constructor
class int_ : public object {
      ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit copy constructor
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:294:29: error: call to constructor of 'pybind11::int_' is ambiguous
        int_ start(start_), stop(stop_), step(step_);
                            ^    ~~~~~
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:273:5: note: candidate constructor
    int_(int value) : object(PyLong_FromLong((long) value), false) { }
    ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:274:5: note: candidate constructor
    int_(size_t value) : object(PyLong_FromSize_t(value), false) { }
    ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit move constructor
class int_ : public object {
      ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit copy constructor
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:294:42: error: call to constructor of 'pybind11::int_' is ambiguous
        int_ start(start_), stop(stop_), step(step_);
                                         ^    ~~~~~
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:273:5: note: candidate constructor
    int_(int value) : object(PyLong_FromLong((long) value), false) { }
    ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:274:5: note: candidate constructor
    int_(size_t value) : object(PyLong_FromSize_t(value), false) { }
    ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit move constructor
class int_ : public object {
      ^
/Users/mik/Desktop/pybind11-master/include/pybind11/pytypes.h:270:7: note: candidate is the implicit copy constructor
3 errors generated.```

Compilation error when trying to use sets

When trying to convert set types, I get a compilation error:

pybind11/include/pybind11/stl.h:87:37: error: invalid conversion from ‘long int’ to ‘PyObject* {aka _object*}’ [-fpermissive]

The source seems to be these lines (starting at line 87):

if (PySet_Add(set, value) != 0) {
    Py_DECREF(value);
    Py_DECREF(set);
    return nullptr;

I think value on lines 87 and 88 should be value_, which is the PyObject * created on line 82:

PyObject *value_ = value_conv::cast(value, policy, parent);

PYBIND11_DECLARE_HOLDER_TYPE with std::shared_ptr

The automatic transparent conversions do not seem to work for std::shared_ptr.

I looked at the example8 and it seems that ref<T> class there differs from shared_ptr in one apparently crucial aspect, it has user-defined conversion

operator T* () { return m_ptr; }

which allows it to be passed as the first argument to cast function.

I managed to get automatic conversion to work with shared_ptr by adding

diff --git a/include/pybind11/cast.h b/include/pybind11/cast.h
index 5fe9342..1cb4ff4 100644
--- a/include/pybind11/cast.h
+++ b/include/pybind11/cast.h
@@ -527,6 +527,12 @@ public:
     explicit operator type&() { return *(this->value); }
     explicit operator holder_type&() { return holder; }
     explicit operator holder_type*() { return &holder; }
+
+    using type_caster<type>::cast;
+    static PyObject *cast(const holder_type &src, return_value_policy policy, PyObject *parent) {
+      return type_caster<type>::cast(src.get(), policy, parent);
+    }
+
 protected:
     holder_type holder;
 };

But perhaps there is a better way?

`#undef WITH_THREAD` not supported

When WITH_THREAD is #undefed in $PYTHON_PREFIX/include/python2.7/pyconfig.h.

./pybind11/include/pybind11/pybind11.h: In function ‘void pybind11::init_threading()’:
./pybind11/include/pybind11/pybind11.h:915:51: error: ‘PyEval_InitThreads’ was not declared in this scope
 inline void init_threading() { PyEval_InitThreads(); }

This problem is also probably in python3.x

How I got here? 3rd party custom python and I cannot change it.

example.so not properly loaded with current branch

Hi,
tried to build on OSX Yosemite,
but it seems something is wrong, example.so doesn't load properly on OSX

__Py_ZeroStruct

is not properly exported...
I'll have a look to see if I can figure out what is going on,
but if anybody has some idea what could be the issue much appreciated...

2016.01.02:19.42][Jiri@JiriMBP:~/lib/cpp/python/pybind11.orig/example]$  python
Python 3.4.3 |Anaconda 2.4.1 (x86_64)| (default, Oct 20 2015, 14:27:51)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import example
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: dlopen(/Users/Jiri/Documents/lib/cpp/python/pybind11.orig/example/example.so, 2): Symbol not found: __Py_ZeroStruct
  Referenced from: /Users/Jiri/Documents/lib/cpp/python/pybind11.orig/example/example.so
  Expected in: flat namespace
 in /Users/Jiri/Documents/lib/cpp/python/pybind11.orig/example/example.so
>>>

System:

Darwin Kernel Version 14.5.0: Tue Sep  1 21:23:09 PDT 2015; root:xnu-2782.50.1~1/RELEASE_X86_64 x86_64 i386 MacBookPro11,2 Darwin

Exception conversion

Boost::python supports, and it would be nice if pybind11 did aswell, conversion of errors. I have legacy code which contains this snippet

if (index < 0 || index >= size())
    throw std::out_of_range("index out of range");

in boost python, this was converted to IndexError in python. This allows my container to be used with python expressions such as list(my_object) since this is implemented something like this in python

def list(obj):
    result = []
    i = 0
    while True:
        try:
            result += [obj[i]]
            i += 1
        except IndexError:
            break

But with pybind11, it becomes a RunTimeError and all such logic fails.

Type caster classes are not exception safe

In general type casters for types that hold multiple values are not exception
safe during conversion from C++ to Python objects, for example:

• std::pair, leaks o1 if first caster succeeds and second caster throws an exception
• std::tuple, leaks already converted tuple elements if subsequent type caster throws an exception
• argument casting in handle::call may leak partially converted arguments

The same is true of casters for std::vector, std::set, std::map.

Why those caster would throw an exception in a first place? There are a few
places where explicit throw is used instead of null return value - I am not
sure if those are accidental or there is supposed to be distinction between
those two situations? In addition, memory allocation in copy_constructor may
throw, or C++ copy constructor itself may throw an exception.

How to write c++ function that can consume python3 `bytes`?

Hello,

This is not necessarily an issue, but a question. Sorry to ask here. I couldn't find an appropriate forum to ask questions about pybind11.

I've a python3 program and I want to rewrite some of the functionality in C++. The function I want to rewrite takes bytes as input. What would be the correct signature for corresponding C++ function? I'm trying to use pybind11 for the binding.

Thanks.

array.API and numpy headers

In array.API, several (but not all) numpy macros are declared as functions, this is all fine, but it causes issues when you import numpy before pybind11.

For example

#include <numpy/arrayobject.h>
#include <pybind11/pybind11.h>

causes compilation crashes because the macros (defined in numpy) interfere with the names defined in pybind11.

From Boost.Python to PYBIND11

Hi,
First off, great job on the library. It looks very promising.

Now to my question. I have a set of functions where each one returns a custom container class called vec. Below is an example wrapper using boost.python with num_util by Phil Austin. Basically num_util takes an iterator and returns a numpy array.

How would I achieve the same results using pybind11?

namespace nbpl = num_util;
namespace bp = boost::python;

bpn::array mybartlett(int L)
{
    // This is a call to my custom function that returns a custom vector
    vec w = bartlett(L);

    // This copies the data in w into the numpy array out
    bpn::array out = nbpl::makeNum(w.begin(), w.size());

    return out;
}

Does not build on GCC 4.9.2-10 / armv7l

I'm getting some errors when building the example with GCC 4.9 on linux. It looks like pybind::ssize_t is actually int on 32-bit, so overloading int_ with both on 32 bit architectures actually triggers an error.

Scanning dependencies of target example
[  7%] Building CXX object CMakeFiles/example.dir/example/example.cpp.o
In file included from /pybind11/include/pybind11/cast.h:13:0,
                 from /pybind11/include/pybind11/pybind11.h:26,
                 from /pybind11/example/example.h:1,
                 from /pybind11/example/example.cpp:10:
/pybind11/pytypes.h:276:5: error: 'pybind11::int_::int_(pybind11::ssize_t)' cannot be overloaded
     int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
     ^
/pybind11/include/pybind11/pytypes.h:273:5: error: with 'pybind11::int_::int_(int)'
     int_(int value) : object(PyLong_FromLong((long) value), false) { }
     ^
CMakeFiles/example.dir/build.make:54: recipe for target 'CMakeFiles/example.dir/example/example.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example.cpp.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/example.dir/all' failed
make[1]: *** [CMakeFiles/example.dir/all] Error 2
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2

use std::decay

Currently we implement decay here, but too me, we should be able to use std::decay. Any reason we don't?

Problems with shared_ptr re-entry

In a project where I am using pybind11 I had a problem with C++ objects wrapped in shared_ptr that were handed over to Python and then re-entered C++. With default approach (using PYBIND11_DECLARE_HOLDER_TYPE) this results in two smart pointers that do not know about each other. Using std::enable_shared_from_this template in my objects I have modified the code generated by PYBIND11_DECLARE_HOLDER_TYPE to something like (basically there is only one small difference at line 7):

template <class T> class type_caster<std::shared_ptr<T>> : public type_caster_holder<T, std::shared_ptr<T>> { 
    typedef type_caster<T> parent;
public:
    bool load(PyObject *src, bool convert) {
        if (!parent::load(src, convert))
            return false;
        holder = ((T *) parent::value)->shared_from_this();
        return true;
    }
    explicit operator T*() { return this->value; }
    explicit operator T&() { return *(this->value); }
    explicit operator std::shared_ptr<T>&() { return holder; }
    explicit operator std::shared_ptr<T>*() { return &holder; }
    using type_caster<T>::cast;
    static PyObject *cast(const std::shared_ptr<T> &src, return_value_policy policy, PyObject *parent) {
        return type_caster<T>::cast(src.get(), policy, parent);
    }
protected:
    std::shared_ptr<T> holder;
};

I hope that this will save some time to anyone who will have the same scenario. If there is a good way to support this by default in some way it would be also very nice. I am not that familiar with C++ templates, but I guess the template could be limited to classes that inherit from std::enable_shared_from_this.

Using pybind11 from pypi - directory with headers.

When using pybind11 from pypi what would be the best way to determine the
directory with include files, in a manner that would work regardless if
pybind11 have been installed system wide, using user scheme, virtualenv, etc?
Maybe you have some suggestion given that pybind11 is distributed on pypi.

I will note, that I like how numpy installation scheme works, which just places
headers along with the python module, making it easy to locate from python:

import numpy.core as core
d = os.path.join(os.path.dirname(core.__file__), 'include')

Though it looks quite differently in pybind11 package (ignoring the fact the
pybind11, doesn't provide any python module).

Convert to object

I want to assign a class, exposed to Python through pybind11, to a module's attribute. This requires converting it to an object. To make things concrete, I want to use RAII pattern to initialize and finalize MPI whenever the module is imported. The code looks roughly like this:

struct environment
{
  environment()    { MPI_Init(0,0); }
  ~environment()   { MPI_Finalize(); }
};

py::class_<environment>(m, "environment")
  .def("__repr__", [](const environment&) { return "MPI Environment"; });

m.attr("env") = py::detail::type_caster<environment>::cast(new environment, py::return_value_policy::automatic, m.ptr());

This works, but it's the last line that bothers me. Is this really the right way to do this? Have I missed some cleaner way? Besides being verbose, having to access detail namespace seems wrong. I'm probably missing something, but if not, maybe it's worth adding clean support for this use-case?

Lubuntu 15.04 Example Build Failures

I cannot compile the examples out of the latest commit (ad7bc01) on my Lubuntu 15.04 machine.

$ gcc --version
gcc (Ubuntu 4.9.2-10ubuntu13) 4.9.2

And the build log:

$ cmake .
-- The C compiler identification is GNU 4.9.2
-- The CXX compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Setting build type to 'MinSizeRel' as none was specified.
-- Found PythonLibs: /usr/lib/i386-linux-gnu/libpython2.7.so (found version "2.7.9")
-- Found PythonInterp: /usr/bin/python (found version "2.7.9")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/pladow/pybind11
$ make -j 4
Scanning dependencies of target example
[ 7%] [ 15%] [ 23%] [ 30%] Building CXX object CMakeFiles/example.dir/example/example2.cpp.o
Building CXX object CMakeFiles/example.dir/example/example3.cpp.o
Building CXX object CMakeFiles/example.dir/example/example.cpp.o
Building CXX object CMakeFiles/example.dir/example/example1.cpp.o
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example3.cpp:10:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example1.cpp:11:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example.cpp:10:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
In file included from /home/pladow/pybind11/include/pybind11/cast.h:13:0,
from /home/pladow/pybind11/include/pybind11/pybind11.h:26,
from /home/pladow/pybind11/example/example.h:1,
from /home/pladow/pybind11/example/example2.cpp:11:
/home/pladow/pybind11/include/pybind11/pytypes.h:254:5: error: ‘pybind11::int_::int_(pybind11::ssize_t)’ cannot be overloaded
int_(ssize_t value) : object(PyLong_FromSsize_t(value), false) { }
^
/home/pladow/pybind11/include/pybind11/pytypes.h:251:5: error: with ‘pybind11::int_::int_(int)’
int_(int value) : object(PyLong_FromLong((long) value), false) { }
^
CMakeFiles/example.dir/build.make:54: recipe for target 'CMakeFiles/example.dir/example/example.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/example.dir/build.make:123: recipe for target 'CMakeFiles/example.dir/example/example3.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example3.cpp.o] Error 1
CMakeFiles/example.dir/build.make:77: recipe for target 'CMakeFiles/example.dir/example/example1.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example1.cpp.o] Error 1
CMakeFiles/example.dir/build.make:100: recipe for target 'CMakeFiles/example.dir/example/example2.cpp.o' failed
make[2]: *** [CMakeFiles/example.dir/example/example2.cpp.o] Error 1
CMakeFiles/Makefile2:60: recipe for target 'CMakeFiles/example.dir/all' failed
make[1]: *** [CMakeFiles/example.dir/all] Error 2
Makefile:86: recipe for target 'all' failed
make: *** [all] Error 2

Function overloading does not seems to work when keyword argument is used

Hi @wjakob,

I noticed that function overloading does not seems to work when keyword argument is used (or works in rather confusing way). Supplying at least one keyword argument (even when no default values is specified) lead to wrong function resolution. Pybind version used: somewhat latest from GitHub (1546b85).

Is this behavior expected? Or something wrong with my code?

Thanks,

Code to replicate this:

#include <pybind11/pybind11.h>
#include <string>

std::string foo(int a) { return "foo1"; }
std::string foo(int a, int b) { return "foo2"; }
std::string foo(int a, int b, int c) { return "foo3"; }

PYBIND11_PLUGIN(example) {
    pybind11::module m("example", "example module");

    m.def("foo", (std::string (*)(int) ) &foo, "doc", pybind11::arg("a"));
    m.def("foo", (std::string (*)(int, int) ) &foo, "doc", pybind11::arg("a"), pybind11::arg("b"));
    m.def("foo", (std::string (*)(int, int, int) ) &foo, "doc", pybind11::arg("a"), pybind11::arg("b"), pybind11::arg("c"));

    return m.ptr();
}

Output:

In [1]: foo(1)
Out[1]: u'foo1'

In [2]: foo(1,2)
Out[2]: u'foo2'

In [3]: foo(1,2,3)
Out[3]: u'foo3'

In [4]: foo(a=1,b=2)
Out[4]: u'foo1'

In [5]: foo(a=1,b=2,c=3)
Out[5]: u'foo1'

Readme say BSD-style license; LICENSE is ZLib

It's not entirely wrong, but calling it the Zlib license in the readme would be a lot clearer.

Warnings: Unused & Offsetof

Thx for the great project! ✨

During compile I get several warnings about:

• unused vars in

./include/pybind11/attr.h(257)
./include/pybind11/attr.h(265)
./include/pybind11/attr.h(269)

• and an other in

./include/pybind11/pybind11.h(566): warning: offsetof applied to non-POD types is nonstandard

Tested with master as of 48548ea, nvcc/7.5.17 & gcc/4.9.2.

What's the best way to deal with different types?

Suppose I want to process an py::dict instance which might contain different types of key/value pairs, i.e. str/int, str/dict. After reading pybind11/pytypes.h, I figure out that such problem could be solved by two steps:

1. invoke handle::get_type to get the type of instance.
2. pass type_handle.ptr() to Py[Type]_Check methods to detect the type.

I'm wondering if there is a better way to resolve the problem, such as function overloading or use some magic code to detect the type of of instance.

Thanks for your work, it's amazing!

basic code gives TypeError: Incompatible function arguments.

Hi

I am using pybind11 to wrap a lot of code, but hit the following problem.

A toy version of the problem is as follows.
Given the following code (hooked it up in the example directory of the pybind11 repo)

#include "example.h"
#include <pybind11/stl.h>

class ExampleJ1 {
public:
    int n{-1};
    ExampleJ1() = default;
    void doSomething( int n_ )  { n = n_; }
};

void init_exJ1(py::module &m) {
    py::class_<ExampleJ1>(m, "ExampleJ1", "Example J1 documentation")
        .def(py::init<>())
        .def("doSomething", &ExampleJ1::doSomething, "set the value n")
        .def_readwrite("n", &ExampleJ1::n, "the value n")
        ;
}

i get the following error

>>> from example import ExampleJ1
>>> e = ExampleJ1
>>> e.n
<property object at 0x1021a5688>
>>> e.n = 1
>>> e.n
1
>>> e.doSomething(1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: Incompatible function arguments. The following argument types are supported:
    1. (example.ExampleJ1, int) -> None

I am at loss, what I m doing wrong here.
Any idea why this doesn't work ?

Thanks
Jiri

Missing specialization for to_string when passing in std::vector<T> as default argument to py::arg

Hi,
I noticed the following compile error:

#include "example.h"
#include <pybind11/stl.h>
#include <vector>

using VecN = std::vector<size_t>;
struct ExampleJ1 {
    VecN nn;
    ExampleJ1( const VecN& nn = VecN{} ) : nn{nn}  {}
};

void init_exJ1(py::module &m) {
    py::class_<ExampleJ1>(m, "ExampleJ1", "Example J1 documentation")
        .def(py::init<VecN>(), py::arg("nn") = VecN() )
        .def_readwrite("nn", &ExampleJ1::nn, "the values nn");
}

gives

[  6%] Building CXX object CMakeFiles/example.dir/example/exampleJ1.cpp.o
In file included from /Users/Jiri/lib/cpp/python/pybind11/example/exampleJ1.cpp:11:
In file included from /Users/Jiri/lib/cpp/python/pybind11/example/example.h:1:
In file included from /Users/Jiri/lib/cpp/python/pybind11/include/pybind11/pybind11.h:26:
In file included from /Users/Jiri/lib/cpp/python/pybind11/include/pybind11/cast.h:13:
In file included from /Users/Jiri/lib/cpp/python/pybind11/include/pybind11/pytypes.h:12:
/Users/Jiri/lib/cpp/python/pybind11/include/pybind11/common.h:202:36: error: no matching function for call to 'to_string'
to_string(const T &value) { return std::to_string(value); }
                                   ^~~~~~~~~~~~~~
/Users/Jiri/lib/cpp/python/pybind11/include/pybind11/pybind11.h:128:49: note: in instantiation of function template specialization
      'pybind11::detail::to_string<std::__1::vector<unsigned long, std::__1::allocator<unsigned long> > >' requested here
        def[entry->keywords++] = strdup(detail::to_string(a.value).c_str());                                                

Adding a specialization for container types to to_string would alleviate the problem.
What would you prefer to do here ?
Regards
Jiri

Property delete option

I was wondering about your thoughts on having def_property (and friends) take a fdel parameter to be called when the property is deleted.

How to deal with structured/record arrays?

Cython provides lightweight conversions between Foo * and a numpy structured array with a corresponding dtype, where Foo is a plain struct. E.g.,

struct Foo {
    uint16_t a;
    double b;
}

mapping to np.dtype([('a', '<u2'), ('b', '<f8')]).

How would you deal with structured arrays in pybind11?

MSVC Warnings

master as of 80a0674 currently generates warnings on MSVC:

• in Debug mode: https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.141/job/c0yeltp07919w9dv https://ci.appveyor.com/project/wjakob/pybind11/build/1.0.141/job/xfv0nem114pjq3fh

NK : warning LNK4075: ignoring '/INCREMENTAL' due to '/LTCG' specification [C:\projects\pybind11\example.vcxproj]

  Generating code
c:\program files (x86)\microsoft visual studio 14.0\vc\include\xatomic.h(1621): warning C4702: unreachable code [C:\projects\pybind11\example.vcxproj]
c:\program files (x86)\microsoft visual studio 14.0\vc\include\xatomic.h(1402): warning C4702: unreachable code [C:\projects\pybind11\example.vcxproj]
  Finished generating code

Documentation about creating numpy arrays

A common use-case for my python extensions is to do some calculations that are difficult to vectorize using numpy. They usually accept some numpy arrays and will also return one but can't be simplified so that py::vectorize can be used. I've found that the documentation could use an example on how to do this. From reading the implementation of py::vectorize in numpy.h, I've managed to assemble the following:

#include <pybind11/pybind11.h>
#include <pybind11/numpy.h>

namespace py = pybind11;

py::array_t<double> add_arrays(py::array_t<double> i, py::array_t<double> j)
{
  py::buffer_info info_i = i.request();
  py::buffer_info info_j = j.request();

  if ((info_i.ndim != 1) || (info_j.ndim != 1))
    throw std::runtime_error("Number of dimensions must be one");

  if (info_i.shape[0] != info_j.shape[0])
    throw std::runtime_error("Input shapes must be equal");

  std::vector<double> ret(info_i.shape[0]);
  for (unsigned int idx = 0; idx < info_i.shape[0]; idx++)
    ret[idx] = ((double*)info_i.ptr)[idx] + ((double*)info_j.ptr)[idx];

  std::vector<size_t> strides = {sizeof(double)};
  std::vector<size_t> shape = {info_i.shape[0]};
  size_t ndim = 1;

  return py::array(py::buffer_info(ret.data(), sizeof(double),
                   py::format_descriptor<double>::value(),
                   ndim, info_i.shape, strides));
}                  


PYBIND11_PLUGIN(test)
{
  py::module m("test", "Testing pybind11 with numpy arrays");

  m.def("add_arrays", &add_arrays, "Adding two numpy arrays");

  return m.ptr();
} 

Furthermore I've found that for 1D-arrays, the return line could be further simplified to

  return py::array(info_i.shape[0], ret.data());

I wonder if this implementation is good or if things could be improved. If I understand correctly, during construction of the numpy array, the memory from the vector is copied. Is it possible to prevent that copy operation somehow?

However for the actual request I think this kind of code would be very interesting to people transitioning from Matlab to Python who want to have a replacement for their MEX extensions. This kind of task (non-vectorizable function on arrays with a returned array) is pretty common there and I spent quite some time figuring it out. While it taught me some interesting things, a good example might save other people from having to do that.

If the above code is good I can also provide a pull request for the docs based on it.

Create imported python object in C++

I'd like to create a Python object from a Python module I import within C++. The code would probably explain it best:

    auto threading = py::module::import("threading");
    py::object Thread = threading.attr("Thread");
    py::function f = py::cast([]() { std::cout << "Hello" << std::endl; });
    m.attr("t") = Thread.call(Py_None, f);

The first three lines work fine, but the last one doesn't.

The error I get (within Python) is:

terminate called after throwing an instance of 'pybind11::cast_error'
  what():  handle::call(): unable to convert input arguments to Python objects

The equivalent Python code works fine:

import threading
def f(): print "Hello"
t = threading.Thread(None,f)

Const correctness

When trying to move from boost::python to pybind11 (I really love this initiative), i learned that pybind11 does not seem to be properly const-correct. This causes issues, for example, one would expect this to work:

void my_fun(const py::array& arr) {
    size_t size = arr.request().count
}

It fails because request is not declared const. The same applies to many other functions.