rosettacommons/binder

Binder, tool for automatic generation of Python bindings

Binder

Binder is a tool for automatic generation of Python bindings for C++11 projects using Pybind11 and Clang LibTooling libraries. That is, Binder, takes a C++ project and compiles it into objects and functions that are all usable within Python. Binder is different from prior tools in that it handles special features new in C++11.

Reference documentation is provided at http://cppbinder.readthedocs.org/en/latest. A PDF version of the manual is available here.

binder's Issues

has anyone gotten this to work on windows 10?

binder build error

Hi:
I'm a beginner of binder.I followed the install guide.when I cmake the source code,some error happened,like this:
"CMake Error: The source directory '/home/yaoyao/myBinder/build' does not appear to contain CMakeLists.txt."
i think i don't miss the step about 'CMakeLists', just like :“ echo 'add_subdirectory(binder)' > /home/yaoyao/myBinder/llvm/tools/clang/tools/extra/CMakeLists.txt“.
i don know why this happened,Can anyone give me a help?
thanks

Using binder on Mac?

Hello. We have some problems using this module.

We're trying to use binder to automatically generate python classes from our C++ code.

We've built binder using the script build.py and it has generated a binder binary.

When we execute that binary to generate the python wrappers, we get this error:

error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
fatal error: 'vector' file not found
#include <vector>

We run it as it's specified here: http://cppbinder.readthedocs.io/en/latest/basics.html
With the std=c++11 option enabled but it seems the compiler does not accept that option and then it does not compile the code as if it were C++.

We're working on Mac and using clang. We tried passing the --compiler clang++ argument to the build.py but it seems that compiler is only used to build binder and then binder uses another compiler to generate the wrapper that can't accept the std=c++ argument?

Does it happen because we are on a Mac architecture?
We've seen that you clone clang within the build folder. Is that the compiler you are using inside binder?
Do you know if binder is compatible with Mac? Maybe there is something that we are missing?

Thanks in advance for your answers.

binder generates almost nothing

i have a qt based library that i try to bind in python.
i specified +class MainClass in binder.config and after running this command:

binder --config binder.cfg --root-module pyqapp --prefix bindings all_includes.hpp \
-- -std=c++11 -fPIC -Wno-c++11-narrowing -Ilibqapp/include -I/usr/include/x86_64-linux-gnu/qt5/ \
-I/usr/include/x86_64-linux-gnu/qt5/QtCore -I/usr/include/x86_64-linux-gnu/qt5/QtGui \
-I/usr/include/x86_64-linux-gnu/qt5/Network -I/usr/include/x86_64-linux-gnu/qt5/QtMultimedia \
-I/usr/include/c++/6/ -I/usr/include/x86_64-linux-gnu/c++/6/

binder generates some empry folder and files, just the main source file contains this:

#include <map>
#include <memory>
#include <stdexcept>
#include <functional>

#include <pybind11/pybind11.h>

typedef std::function< pybind11::module & (std::string const &) > ModuleGetter;



PYBIND11_PLUGIN(pyqapp) {
	std::map <std::string, std::shared_ptr<pybind11::module> > modules;
	ModuleGetter M = [&](std::string const &namespace_) -> pybind11::module & {
		auto it = modules.find(namespace_);
		if( it == modules.end() ) throw std::runtime_error("Attempt to access pybind11::module for namespace " + namespace_ + " before it was created!!!");
		return * it->second;
	};

	modules[""] = std::make_shared<pybind11::module>("pyqapp", "pyqapp module");

	std::vector< std::pair<std::string, std::string> > sub_modules {
	};
	for(auto &p : sub_modules ) modules[p.first.size() ? p.first+"::"+p.second : p.second] = std::make_shared<pybind11::module>( modules[p.first]->def_submodule(p.second.c_str(), ("Bindings for " + p.first + "::" + p.second + " namespace").c_str() ) );


	return modules[""]->ptr();
}

c++11 noexcept handling

Currently I have a project that needs to utlize noexcept of c++11

The line of interest is here:

binder/binder/class.cpp

Line 482 in 814d7c3

c += "\t{} {}({}){} override {{ "_format(return_type, m->getNameAsString(), std::get<0>(args), m->isConst() ? " const" : "");

After some digging I was still not able to find how I can fetch if the method decl has noexcept or not -- is it an attribute of the decl?

Don't include standard libraries?

It looks like a TON of standard libraries are getting written to every binding file, and multiple times. Is there an option to include only the headers needed in the file?

I'm using VS 2015.

Compilation script doesn't work in my case

I'm trying to compile binder. I'm following the script from documentation:
https://cppbinder.readthedocs.io/en/latest/install.html
but:

1. llvm required in the third stage is not $HOME/prefix/llvm/; I think the very first line of that script should be:
   cd $HOME/prefix/
   and llvm should be cloned here, right?

2. the symlink step doesn't work:
   ln -s $HOME/binder/binder $HOME/prefix/llvm/tools/clang/tools/extra/binder
   because there is no directory binder/binder anywhere. Should I use binder/source instead?

3. Finally according to the script cmake should be executed in $HOME/prefix/build. This directory is empty (which is normal) but the directory one level up ($HOME/prefix) is also empty. So where should cmake be executed?

Error: expected unqualified-id before numeric constant

I just ran in the following issue fmtlib/fmt#398.
With fmt master at fmtlib/fmt@d49f206 or version 3.0.1 I was able to successfully build binder.

std::functional argument generates incorrect code

The following code fails to generate pybind11 code that can compile.

#ifndef THING_H
#define THING_H
#include <functional>
namespace work{
int a(std::function<int(double, int)> fn) {
  return fn(1.0, 2);
}
}
#endif

The errors are all similar to this:

std/functional.cpp:43:61: error: expected expression
                pybind11::class_<std::_Maybe_unary_or_binary_function<int,<const double &, int>>, std::shared_ptr<std::_Maybe_unary_or_binary_function<int,<double, int>>>, std::binary_function<co...
                                                                          ^

The problem is the second template argument to std::_Maybe_unary_or_binary_function<int,<double, int>> should not have the angle brackets around the arguments (it accepts variadic template). If I manually strip the brackets, the code compiles.

Wrong documentation about return_policy

Found old(wrong) notes about default_pointer_return_value_policy and etc
http://cppbinder.readthedocs.io/en/latest/config.html
From source code here no such option, but exist default_member_pointer_return_value_policy
and etc.
Please fix documentation

Binder missing functions

I have built binder from github. I have tried to build pybind11 code using:

binder --root-module test --bind "work" binder.cpp -- -std=c++11

for the C++:

#ifndef THING_H
#define THING_H
#include <vector>
namespace work{
struct Thing {
double value;
};
std::vector<Thing> get();
//inline std::vector<Thing> get2() { return std::vector<Thing>(); };
}
#endif

However, the function get does not get created unless I uncomment the last function get2. Am I doing something wrong?

Which classes and functions are left out?

Thanks for the nice tool, seems to be a good start at automatic binding generation.

Binder seems to do nicely for some of my classes, but most of them are not included in the output files. What are the reasons for binder to decide which classes can be exported and which not? Would be nice to have more documentation on this, or some output while processing the files!

Trampoline classes causes compiler warning: violates the C++ One Definition Rule [-Wodr]

Assume we have following hierarchy:

namespace n1 {
  class my_exception : public std::exception {}
  namespace n2 {
    class my_exception: public ::my_exception {}
  }
}

cppbinder will generate a trampoline structures:

struct PyCallBack_my_exception : public n1::my_exception {}

struct PyCallBack_my_exception : public n1::n2::my_exception {}

And even worse if situation will be the same as in this example. There will be an error because of type redefinition.
In our project we have such types in separate modules so we have only warnings.

Is the new Binder with latest pybind11 working?

Is the new version of Binder to support Pybind11 Module instead of plugins working? I tried compiling my library using the generated binding file of new Binder. But it resulted it errors. When I performed a diff between the versions of Generated cpp files (my previouse working versus newly generated binding cpp) I could see some changes.

Operating system is Ubuntu 14.04 and the compiler is clang++ 4.0.1

./chenhancc/includes/pybind11/include/pybind11/cast.h:1380:5: error: static_assert failed "Holder classes are only supported for custom types"
    static_assert(std::is_base_of<base, type_caster<type>>::value,

Regards,

#0K

Build with Visual Studio 2015

I generated the project files using cmake and then build the "INSTALL" solution. A lot of executables get built but binder.exe is nowhere to be found. I used the clang platform tools for the binder project to avoid compile errors. Where/when is the binder executable suppose to be built?

Distributing with setup.py

Firstly, I feel guilty to post this as an issue. Nope this is not an issue. I was able to compile the library successfully and got an .so generated. It was faster like lightning inside python. Really happy with the results I aimed for.

The last step would be to know how can I distribute this to users via pip or easy_install? I have a python script that takes in the path information related to the libraries and include files. Then running python compile_binder.py creates the python library from the c++ code and generates the library from the binding code.

I can see even the generation of binding files are not necessary as they can be done once and redistributed along with the source files. But clang++ is needed to compile the binding file to a library. Anyone any ideas how you did this for your project?

Regards,

#0K

Adding code in output main file

How i can add code (initialize global static variable of my lib) in output file?

Turn off exporting operator and program structure with multiple modules

Part 1:
In the very simple test cases I've put together I see that Binder by default exports the assignment operator for a simple Rectangle class. I don't explicitly define it in C++ (maybe I should), but I can't compile the bindings with it in there and they seem to work ok when I comment it out. Is this bad practice from my end? Or should there be a way to tell Binder not to generate bindings for operators?

Part 2:
Eventually I'd like to tackle wrapping a large C++ project that will be made up of multiple interconnected modules. So in Python it will be something like:

from root-module.mod1 import Class1, ...
from root-module.mod2 import Class2, ...
from root-module.mod3 import Class3, ...

It looks like this can be accomplished using pybind11:module::import() before you declare a new module. Is there native functionality for this in Binder? It seems every dependency gets put in the source file.

Thanks,
Trevor

Update:
As a semi-automated first attempt at this, maybe there can be an option that binder will only generate bindings for items defined in all_includes.h (it won't go recursing into those headers and write bindings). And then in the config file you could have a +import modname flag that would insert the pybind11::module::import(modname)?

Build instructions

The Build Instructions should mention that ninja is needed for compiling binder.

The provided shell sequence does not work. Checking out clang releases requires origin/... before the branch name, and the cmake command does not find its list file in the provided path.

Also, I first tried to follow the shell sequence, before I saw that there is a binder/install.py, which make it easier. This could be highlighted a bit better ;-)

Prefix path

In order to get output files, I have to use a binder --prefix path/to/out/ path that ends with a slash. This is not documented, and could be fixed easily ;-)

error in example

Running into issue creating shared library from the example. Please refer to https://cppbinder.readthedocs.io/en/latest/examples.html#building-bindings-basics, on step 5.

By running the command:

g++ -o test_struct.so -shared test_struct/test_struct.o test_struct.o

It yields:

test_struct.o: file not recognized: File format not recognized
collect2: error: ld returned 1 exit status

Enum arithmetic

Pybind11 supports exporting enums to be used in arithmetic operations.
cppbinder does not.
For example we have enum with fields containing a bit shifted values to be used as a flags:

enum Flags {
  flag1 = 1,
  flag2 = 1 << 1,
  flag3 = 1 << 2
}

Using pybind11 e may write:

pybind11::enum_<Flags>(m, "Flags", pybind11::arithmetic())
  .value("flag1", Flags::flag1)
  .value("flag2", Flags::flag2)
  .value("flag3", Flags::flag3)
  .export_values();

Also there are no possibility to provide custom binding for enums. Nor to exclude enum from generation (for example specifying -enum in config).

Installation stucks when exexcuting the build-and-run-tests.py

Just like what you see in ths screen shot.

Minor: Command line options doc

In the documentation for the command line options , the dash symbol – is a single wide dash for all options, instead of the needed two normal dashes --. Thus, copy-paste from the documentation does not work.

Header includes

The documentation does not mention that my project's #include directives have to use <...> instead of "...". This could me mentioned somewhere ;-)

unresolved external symbol in class?

I get this error when I try and build LLVM/clang/binder:

I could never get the build.py script to work so I grabbed the latest LLVM/clang repo and manually put the binder folder in the file structure. I've applied the changes as described here.

I'm using Windows 10 64-bit Visual Studio 2015 Community Edition.

Update:
I get the same result for latest LLVM/Clang (version 5) and hard reset to 3.8.

One Constructor with default values instead of multiple ctors with different number of arguments

Hi. I am just wondering have you tried creating one constructor with default arguments the same way as it is described in pybind11 documentations??

I tried to do that. I did not find a correct way that clang stuff may support for default value of the argument. So I did it in this way:
https://github.com/RosettaCommons/binder/compare/master...yosoufe:defaultValueForDefaultArguments?expand=1
I have already seen some problem with it that it does not generalize. for example if the argument type is Enum, in the C++ code the Enum should be defined with its all namespaces until it will be compilable.

For example:

struct Astruct{
  enum enumType{
    T1,
    T2
  }
  enumType t;
  double value1;
  Astruct(double v1, enumType type = T1);
}

is compilable. But in my branch, I have to write it in the following way until the bindings are also compilable Astruct(double v1, enumType type = Astruct::enumType::T1); because I simply copy what is after the = into the binding codes.

Do you have any better idea??
And also any other problems that may occur that I cannot see it?

Compiler errors

After I produced the bindings files, I now tried to compile everything, but there are a couple of problems. I am not entirely sure yet where they come from - my part, Pybind, or Binder. It seems, the latter, so I'm opening this issue.

Clang 3.9.1, WARN_FLAGS "-Wall -Wextra -Wno-unknown-pragmas -pedantic -pedantic-errors"

In file included from [...]/src/placement/function/functions_1.cpp:23:
In file included from [...]/pybind11/include/pybind11/pybind11.h:36:
In file included from [...]/pybind11/include/pybind11/attr.h:13:
[...]/pybind11/include/pybind11/cast.h:961:79: error: token pasting of ',' and __VA_ARGS__ is a GNU
      extension [-Werror,-Wgnu-zero-variadic-macro-arguments]
    struct always_construct_holder<holder_type> : always_construct_holder<void, ##__VA_ARGS__>  { }; \
                                                                              ^
[...]/pybind11/include/pybind11/cast.h:961:79: error: token pasting of ',' and __VA_ARGS__ is a GNU
      extension [-Werror,-Wgnu-zero-variadic-macro-arguments]
...

The first line functions_1.cpp:23 is simply #include <pybind11/pybind11.h>.

The warning comes from a GNU extension. The interesting part is that all the includes (attr.h, cast.h) are part of the pybind header. So they should also happen when I try the minimal example of Pybind - but there, I don't get any errors. The minimal example works for me.

So, I tried to disable the warnings by leaving out the pedantic flag. But then, other errors occur:

Clang 3.9.1, WARN_FLAGS "-Wall -Wextra -Wno-unknown-pragmas"

In file included from [...]/src/placement/function/helper.cpp:1:
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/hashtable_policy.h:34:15: error: expected '{'
namespace std _GLIBCXX_VISIBILITY(default)
              ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/hashtable_policy.h:34:15: error: C++ requires a type
      specifier for all declarations
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/hashtable_policy.h:34:35: error: expected expression
namespace std _GLIBCXX_VISIBILITY(default)
                                  ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/hashtable_policy.h:34:43: error: expected ';' after top
      level declarator
namespace std _GLIBCXX_VISIBILITY(default)
                                          ^
...

Here, the first line helper.cpp:1 is #include <bits/hashtable_policy.h>.

It seems a bit weird to me that Binder directly includes bits headers instead of using the "clean" names of those headers. But maybe this is needed. Anyway, does not seem to work, so I tried GCC instead.

GNU 4.9.4, WARN_FLAGS "-Wall -Wextra -Wno-unknown-pragmas -pedantic -pedantic-errors"

[...]/src/placement/function/functions_1.cpp:27:52: error: ISO C99 requires rest arguments to be used
  PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
                                                    ^
[...]/src/placement/function/functions_1.cpp:28:36: error: ISO C99 requires rest arguments to be used
  PYBIND11_DECLARE_HOLDER_TYPE(T, T*);
                                    ^
[...]/src/placement/function/functions_1.cpp:27:53: error: extra ‘;’ [-Wpedantic]
  PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
                                                     ^
[...]/src/placement/function/functions_1.cpp:28:37: error: extra ‘;’ [-Wpedantic]
  PYBIND11_DECLARE_HOLDER_TYPE(T, T*);
                                     ^
...

The line that is causing this is the definition PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);, as part of the #ifndef BINDER_PYBIND11_TYPE_CASTER block. So, this seems to be a similar problem as the first one with Clang. Next, naturally, I tried again without pedantic:

GNU 4.9.4, WARN_FLAGS "-Wall -Wextra -Wno-unknown-pragmas"

In file included from [...]/src/placement/function/helper.cpp:1:0:
/usr/include/c++/4.9/bits/hashtable_policy.h:34:15: error: expected ‘{’ before ‘_GLIBCXX_VISIBILITY’
 namespace std _GLIBCXX_VISIBILITY(default)
               ^
/usr/include/c++/4.9/bits/hashtable_policy.h:34:34: error: expected constructor, destructor, or type conversion before ‘(’ token
 namespace std _GLIBCXX_VISIBILITY(default)
                                  ^
In file included from /usr/include/c++/4.9/typeinfo:34:0,
                 from /usr/include/c++/4.9/debug/formatter.h:34,
                 from /usr/include/c++/4.9/debug/functions.h:41,
                 from /usr/include/c++/4.9/debug/debug.h:127,
                 from /usr/include/c++/4.9/bits/unique_ptr.h:34,
                 from [...]/src/placement/function/helper.cpp:2:
/usr/include/c++/4.9/exception:35:37: error: expected ‘}’ before end of line
 #pragma GCC visibility push(default)
                                     ^
...

Similar to the second Clang problem.

Any help is appreciated. What is going on here? Thanks in advance!

Missing inherited constructor

When I have a class that inherits a constructor with the "using" phase, it is missing in the pybind11 code.

I think this is similar to issue #39 where here clang is optimising the constructor out. The only way I have been able to include it is by using the constructor in the header with const b = B(1); but the bigger issue there could be many constructors etc.

Any advice?

#ifndef THING_H
#define THING_H
namespace work{
class A
{
public:
explicit A(int x) {}
};
class B: public A
{
using A::A;
};
}
#endif

Prebuilt binaries for Windows? Possible?

Is it possible to have pre-built binaries of binder for Windows 7? I am not sure if it is too much to ask for. I found it was a swift breeze building binder on an unix system. But with Windows, Visual Studio, ATL libraries everything is a royal pain everywhere in the body.

Regards,

#0K

problem in std vector of pointer as argument

Hi. In binding of something like the following

Class B { // Virtual Base class
    virtual void f()=0;
}
Class A
{
    ....
    void someFunction(const char* s, std::vector<B*> bs) {....}
}

I get the following binding code:

cl.def("someFunction", (void (A::*)(const char *, int)) &A::someFunction);

Do you know why the std::vector<B*> is turned to int in the second argument of someFunction??

Of course I cannot compile the binding codes.

Tests failing

I wanted to play a bit with the library, but I'm running into problems when running the tests.

I'm running on Ubuntu 16.04:

sancho@united ~ uname -a
Linux united 4.15.0-29-generic #31~16.04.1-Ubuntu SMP Wed Jul 18 08:54:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

I cloned the repository and ran python build-and-run-tests.py -j8, which resulted in the following error:

sancho@united: ~/src/binder python build-and-run-tests.py -j8                                               
Building tool: binder...
cd /home/sancho/src/binder/build/llvm-4.0.0/build_4.0.0.linux.united.release && cmake -G Ninja -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_EH=1 -DLLVM_ENABLE_RTTI=ON  .. && ninja bin/binder clang -j8

Checking out PyBind11 revision: 2d0507db43cd5a117f7843e053b17dffca114107...
cd /home/sancho/src/binder/build/pybind11 && ( ( git fetch && git reset --hard 2d0507db43cd5a117f7843e053b17dffca114107 ) )

Running Binder self-test script...
cd /home/sancho/src/binder/test && ./self-test.py --binder /home/sancho/src/binder/build/llvm-4.0.0/build_4.0.0.linux.united.release/bin/binder --pybind11 /home/sancho/src/binder/build/pybind11/include
Removing old test dir /home/sancho/src/binder/test/build/ ...
T00.basic.hpp Running test...
/home/sancho/src/binder/build/llvm-4.0.0/build_4.0.0.linux.united.release/bin/binder --bind "" --root-module T00_basic --prefix /home/sancho/src/binder/test/build/ --single-file --annotate-includes  /home/sancho/src/binder/test/build//T00.basic.hpp.include -- -x c++ -std=c++11 -I /home/sancho/src/binder/test -I /home/sancho/src/binder/test/..
Generate bindings, pass 1...
Generate bindings, pass 2...
Sorting Binders...
Sorting Binders... Done.
Writing code...
Writing code... Done.

T00.basic.hpp Compiling binding results...
cd /home/sancho/src/binder/test/build/ && clang++ -O3 -shared -std=c++11 -I /home/sancho/src/binder/build/pybind11/include -I/usr/include/python2.7 -I./.. -I./../.. -I./../../source T00_basic.cpp -o T00_basic.so -fPIC
In file included from T00_basic.cpp:9:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/pybind11.h:43:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/attr.h:13:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/cast.h:13:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/pytypes.h:12:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/detail/common.h:111:
In file included from /usr/include/python2.7/Python.h:85:
/usr/include/python2.7/unicodeobject.h:534:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register PyObject *obj,     /* Object */
    ^~~~~~~~~
/usr/include/python2.7/unicodeobject.h:553:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register PyObject *obj      /* Object */
    ^~~~~~~~~
/usr/include/python2.7/unicodeobject.h:575:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register const wchar_t *w,  /* wchar_t buffer */
    ^~~~~~~~~
/usr/include/python2.7/unicodeobject.h:593:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register wchar_t *w,        /* wchar_t buffer */
    ^~~~~~~~~
In file included from T00_basic.cpp:9:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/pybind11.h:43:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/attr.h:13:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/cast.h:13:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/pytypes.h:12:
In file included from /home/sancho/src/binder/build/pybind11/include/pybind11/detail/common.h:111:
In file included from /usr/include/python2.7/Python.h:94:
/usr/include/python2.7/stringobject.h:173:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register PyObject *obj,     /* string or Unicode object */
    ^~~~~~~~~
/usr/include/python2.7/stringobject.h:174:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register char **s,          /* pointer to buffer variable */
    ^~~~~~~~~
/usr/include/python2.7/stringobject.h:175:5: warning: 'register' storage class specifier is deprecated and incompatible with C++17 [-Wdeprecated-register]
    register Py_ssize_t *len    /* pointer to length variable or NULL
    ^~~~~~~~~
7 warnings generated.

T00.basic.hpp Testing imports...
cd /home/sancho/src/binder/test/build/ && python2.7 -c 'import T00_basic'

Comparing results for test T00.basic.hpp...
diff /home/sancho/src/binder/test/T00.basic.ref /home/sancho/src/binder/test/build//T00_basic.cpp
9a10
> #include <functional>


Encounter error while executing: diff /home/sancho/src/binder/test/T00.basic.ref /home/sancho/src/binder/test/build//T00_basic.cpp

Encounter error while executing: diff /home/sancho/src/binder/test/T00.basic.ref /home/sancho/src/binder/test/build//T00_basic.cpp
9a10
> #include <functional>



Encounter error while executing: cd /home/sancho/src/binder/test && ./self-test.py --binder /home/sancho/src/binder/build/llvm-4.0.0/build_4.0.0.linux.united.release/bin/binder --pybind11 /home/sancho/src/binder/build/pybind11/include

Encounter error while executing: cd /home/sancho/src/binder/test && ./self-test.py --binder /home/sancho/src/binder/build/llvm-4.0.0/build_4.0.0.linux.united.release/bin/binder --pybind11 /home/sancho/src/binder/build/pybind11/include

Any ideas what could be wrong?

fatal errors from clang although the bindings are correct

Hi,
I am trying to make bindings for a single file in a large project. I get the clang file not found fatal error for the included header files which are only used in private sections of my classes and the their path are not included using -I in arguments. It seems that the generated bindings are correct. but the fatal error is annoying.
I do not know clang and ClangTool and ...
How can I suppress these fatal errors?
Is it possible to use something like clang::Preprocessor::SetSuppressIncludeNotFoundError?

Support for nested classes

It seems binder doesn't bind nested classes. Is there an option for this? I'm not a C++ expert, but I think this demonstrates the issue:

#ifndef _INCLUDED_T12_nestedclass_hpp_
#define _INCLUDED_T12_nestedclass_hpp_

class A
{
public:
    
    int x = 1;

    class B {
        public:
        int y = 1;
    };
};

#endif // _INCLUDED_T12_nestedclass_hpp_

results in the bindings...

#include <T12.nested_class.hpp> // test3::A
#include <iostream> // --trace
#include <sstream> // __str__

#include <pybind11/pybind11.h>
#include <pybind11/functional.h>


#ifndef BINDER_PYBIND11_TYPE_CASTER
	#define BINDER_PYBIND11_TYPE_CASTER
	PYBIND11_DECLARE_HOLDER_TYPE(T, std::shared_ptr<T>);
	PYBIND11_DECLARE_HOLDER_TYPE(T, T*);
	PYBIND11_MAKE_OPAQUE(std::shared_ptr<void>);
#endif

void bind_T12_nested_class(std::function< pybind11::module &(std::string const &namespace_) > &M)
{
	std::cout << "B16_[test3::A] ";
	{ // test3::A file:T12.nested_class.hpp line:16
		pybind11::class_<test3::A, std::shared_ptr<test3::A>> cl(M("test3"), "A", "");
		pybind11::handle cl_type = cl;

		cl.def(pybind11::init<>());
		cl.def_readwrite("x", &test3::A::x);
	}
}

Which doesn't seem to bind class B or its parameter y.

Thanks for the support.

Minor: Doxygen comments

Most of my functions and classes are documented with Doxygen-style comments, e.g.

/**
 * @brief Stream interface for reading data from an @link BaseInputSource InputSource@endlink,
 * that keeps track of line and column counters.
 *
 * This class provides...
 */

Binder generates this doc string:

"Stream interface for reading data from an \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n"

It is cool that Binder actually takes care of "binding" the documentation, too! However, it seems to break at @... directives other than @brief. Maybe, it is not an important feature to fix, because being able to process Doxygen directives might be too much, So in the future, I might work on this in a fork, but for completeness, I wanted to report it here, too, and get your opinion on this.

Test reference values are inconsistent in some tests on ubuntu 16.04.

I'm seeing inconsistent reference values when running on our ubuntu server:

test/T10.inheritance.ref
test/T11.override.ref
test/T40.stl.ref
test/T42.stl.names.ref

Host info:

fordas@digs:~/workspace/binder_update_1710/binder$ cat /etc/*release*
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04.2 LTS"
NAME="Ubuntu"
VERSION="16.04.2 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.2 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

Windows compile issue

Dear Wizards,

I was able to use Binder successfully and produce a working python library for Linux boxes. The link to my project is here. Now I am struggling to get the build process work with Windows machines. I have access to a windows 10 machine and that's where my testing is done for windows using Visual Studio 2017. I can say that my library is minimal and does not depend on any external libraries.

When I try to run python setup.py build I get the errors as seen in the attachment
winbuild.log. The primary error is about error C2039: '_Rb_tree_const_iterator': is not a member of 'std'. Am not sure fixing this will fix other errors as well.

Anyone any ideas where things might be going wrong?

Regards,

#0K

Support for operator>> and operator<<

binder/source/function.cpp

Line 42 in 0333a4d

static std::map<string, string > const cpp_python_operator_map {

Are there any reasons why operators like '>>', '<<' are not included in this list?
Pybind11 may deal with these.

Remove default constructor for struct s

Is there a way to disable generation of default constructor for the structs??

I have a abstract struct with a pure virtual copy() function. The bindings for the type should be generated but without any constructor. I added something like -function A::A (for struct A) to the config but still the constructor is created and I can create the object in python.

Binder includes in singlefile

If i use option '-single-file' then

#include <map>
#include <memory>
#include <stdexcept>
#include <functional>

#include <pybind11/pybind11.h>

placed before PYBIND11_PLUGIN function , but after other modules code and cant be used.
Please move this includes to top of singlefile when generate single file.

Compilation error on Windows Visual Studio 2015

Hello, I found some compilation errors on my environment.

1. and, or, not was not defined, perhaps some macros these are missing.

#define and &&
#define or ||
#define not !

https://stackoverflow.com/questions/555505/when-were-the-and-and-or-alternative-tokens-introduced-in-c/555524#555524
According to this, need to add another compilation option Za in case of VC++.

2. util.cpp needs to have #include since it has std::isalpha.

3. context.cpp has invalid symbol(?) in the log like this.
   outs() << "Generate bindings, pass №" << pass << "...\n";

test error：╮(╯▽╰)╭

Encounter error while executing: diff /home/yubing/Documents/binder/test/T01.enum.ref /home/yubing/Documents/binder/test/build//T01_enum.cpp
37c37
< cl.def(pybind11::init<>());

  cl.def( pybind11::init( [](){ return new A(); } ) );

Only one add_on_binder for a type is allowed

I want to re-use binders for multiple types. But also use several binders for some type.
Like following:

+add_on_binder my::type binder1
+add_on_binder my::type binder2

And expecting that both binders will be applied.
But actually only last one is used.

From config.cpp source:

 } else if( token == _add_on_binder_ ) {

         if(bind) {
                 auto binder_function = split_in_two(name, "....");
                 add_on_binders_[binder_function.first] = binder_function.second;
         }
 }

It is seen that binder function is overridden each time for type.

Is this behaviour intentional? Or this is a bug?

Inheritance problems

Dear everyone,

I have been trying to pport a C++ library to python and discovered the dynamic duo's Pybind11 and Binder. Though the time to compile binder was longer but still it was worth it.

Getting straight to the problem, I have the following files

X.hpp
A.hpp
B.hpp

all the above are compiled in the namespace "hello"

//contents of X.hpp
struct X
{
	double x, y, z;
	X();
	X(double x, double y, double z);
};


//contents of A.hpp
#include <vector>
#include <X.hpp>

class A
{
	public std::vector<X> m_contents;
	A();
	void addContents(std::vector<X> contents);
};

void A::addContents(std::vector<X> contents)
{
	for (X item : contents)
	{
		m_contents.push_back(X(item.x, item.y, item.z))
	}
}

//contents of B.hpp
class B: virtual public A
{
	B();
};


in Python (version 3.5)
---------------------------------

>>>from hello import X, A, B;
>>>from random import random;
>>>contents = [X(random(), random(), random()) for i in range(20)];
>>>aa = A();
>>>bb = B();
>>>aa.addContents();
>>>#It compiles fine until this point. After this the sub class instance does the same and crashes
>>>bb.addContents();
>>>Segmentation Fault and crashes as seen in the below message
>>>12139 Segmentation fault      (core dumped) python3

---

Can anyone point me the silly mistake I am making here? Or it it a genuine issue?

Regards,

#0K

Analyze code failed on msvc project

os: windows 7 x64
compilers: msvs 2015 sp3, clang 4.0.0 x86 (downloaded as binaries from official clang site)
I just compile binder via clang 4.0.0 x86 (using patched sources and include 'ciso646' #3) and take binder.exe.
now when i try use it on my msvc project then take errors

e:\code\zog\pyunitylib.git\extras>binder --root-module pyunitylib --prefix ../src --config ./pyunitylib.binder.conf py_unity_unpacker_lib.h -- -std=c++11 -Ie:\code\zog\unity-unpacker-lib.git\include -Ie:\code\zog\unity-unpacker-lib.git\unity_unpacker_lib -Ie:\code\zog\unity-unpacker-lib.git -DNDEBUG
error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
In file included from e:\code\zog\pyunitylib.git\extras\py_unity_unpacker_lib.h:1:
In file included from e:\code\zog\unity-unpacker-lib.git\unity_unpacker_lib\unity_unpacker_lib.h:4:
In file included from e:\code\zog\unity-unpacker-lib.git\plugins/plugin.h:3:
In file included from e:\code\zog\unity-unpacker-lib.git\include\Pluma/Pluma.hpp:32:
In file included from e:\code\zog\unity-unpacker-lib.git\include\Pluma/Provider.hpp:32:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\string:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\istream:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\ostream:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\ios:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocnum:7:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\cmath:647:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xtgmath.h:8:
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xtr1common:213:22: error: use of undeclared identifier 'char16_t'
struct _Is_integral<char16_t>
^
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xtr1common:219:22: error: use of undeclared identifier 'char32_t'
struct _Is_integral<char32_t>
^
In file included from e:\code\zog\pyunitylib.git\extras\py_unity_unpacker_lib.h:1:
In file included from e:\code\zog\unity-unpacker-lib.git\unity_unpacker_lib\unity_unpacker_lib.h:4:
In file included from e:\code\zog\unity-unpacker-lib.git\plugins/plugin.h:3:
In file included from e:\code\zog\unity-unpacker-lib.git\include\Pluma/Pluma.hpp:32:
In file included from e:\code\zog\unity-unpacker-lib.git\include\Pluma/Provider.hpp:32:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\string:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\istream:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\ostream:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\ios:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocnum:10:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\streambuf:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xiosbase:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xlocale:8:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\stdexcept:6:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\exception:7:
In file included from C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\type_traits:6:
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xstddef:404:14: error: use of undeclared identifier 'char16_t'
struct hash<char16_t>
^
C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\INCLUDE\xstddef:410:14: error: use of undeclared identifier 'char32_t'

and etc
I not familar with llvm and clang but seems binder use clang analyzer but not pass option "-fms-compatibility-version=19".
how i can pass this option via command option or from source?

Can't bind simple call-back

It was added a support for std::function.
But when trying to bind a code like following:
void set_callback(std::function(void()) callback)

the binder is crashing on assert (if built as Debug):
build/llvm-4.0.0/tools/clang/include/clang/AST/TemplateBase.h:251: clang::QualType clang::TemplateArgument::getParamTypeForDecl() const: Assertion `getKind() == Declaration && "Unexpected kind"' failed.

Or skips this function if built as Release.

While debugging I have found that it is crashing in class.cpp:is_std_function_bindable()

May suspect that the issue will be also with a call-backs uses built-in types like:
int(); void(int); etc.

Include path

The documentation should mention that the include path bind -- -Ipath needs to be absolute (at least on Ubuntu) to work.

Binder appears to write main cpp file before checking if output directory exists

This is a weird edge-case that might be a confluence of a number of things.

Basically, I have a project built as follows

#!/bin/bash

rm -rfv /media/sf_Code/scfft/libraries/sndlib_py/generated_binding/
sync

/home/durr/binder/build/llvm-4.0.0/build_4.0.0.linux.desktest.release/bin/binder \
	-v \
	--root-module sndlib_py                                                       \
	--prefix /media/sf_Code/scfft/libraries/sndlib_py/generated_binding/         \
	--config binder_config.config                                                \
	--bind "my_namespace"                                                               \
	binder_includes.hpp                                                          \
	--                                                                           \
	 -std=c++11                                                                  \
	-I/media/sf_Code/scfft                                                       \
	-DNDEBUG

If I have the rm command remove everything including the generated_binding directory, the generator runs without error and produces the expected output for everything, but sndlib_py.cpp never gets written. However, if I change the rm command to not delete the directory:

rm -rfv /media/sf_Code/scfft/libraries/sndlib_py/generated_binding/*

sndlib_py.cpp gets successfully written.

There's a bunch of confounding factors, but it appears that the writing of at least some of the output files is not properly checked for success. Additionally, verbose mode doesn't ever mention the writing of sndlib_py.cpp, though it does mention all the other generated *.cpp files.

I'm working in a virtualbox VM which is mounting a host directory, so there could be shared folder oddness going on, but the issue seems to be quite reproducible.

Fatal crash during binder generation

Is there any way to get diagnostic output when I get a fatal crash?

I didn't build in debug mode, I'll try that next, but just wondering if there is a verbosity flag or anything to see what it's working on before/during crash. It reads the thousands of header files in all_includes.h, but then crashes during the binder generation before writing any output files. Based on some other issues I've read here, my guess it has something to do with errors in my header files (forward declarations and order of include files).

final class workaround

Is there a workaround for final class? Currently the tool generates a subclass which cannot compile