example 10.2 not working with soqt

[coin3d]

example 10.2 not working with soqt

[looooo]

looooo@246b859
I get this output with the added example:

Traceback (most recent call last):
<0x7ffd587590c0  at 0x7fbf24eaef80>
  File "/home/lo/projects/pivy/examples/soqt/renderAreaCallback.py", line 9, in foo
    print(event.button())
AttributeError: 'PySide.QtCore.QEvent' object has no attribute 'button'

[InventorMentor]

I found the following bug in soqt.i: Function unwrapInstance is only available in sip, but not in shiboken! In the latter, it is called getCppPointer. Therefore, it should be
shiboken_unwrapinst_func = PyDict_GetItemString(PyModule_GetDict(shiboken), "getCppPointer");

Does this already fix this issue for you on linux64? There is at least one additional bug specific two win64, though, a 32->64 bit address overflow problem with wrapInstance using the official shiboken 1.2.2 because with MSVC even on 64 bit systems an unsigned long is still mapped there to an uint32, not an uint64 (in MSVC this corresponds to an unsigned long long). Since win64 assigns most addresses for applications beyond 4 GB now, this results in an address overflow problem. Cf.
https://bugreports.qt.io/browse/PYSIDE-103
http://lists.qt-project.org/pipermail/pyside/2013-September/001569.html

I'm going to retry whether this fixes my remaining issue(s) with a correspondingly patched version of Shiboken.

[looooo]

Thanks for the information. I tested only with python2 because only there I have a shiboken available. The unwrapInstance was in the file used for python3. Should be updated now.

But the problem with returning the correct subclass of QEvent is still remaining. Yesterday I figuered out, that the typemaps(out) for QEvent are not placed inside the wrapper-cpp-file. The only function which has potential to be the root of the problem for 10.2 can be found here SoQtRenderAreaEventPythonCB(void * closure, QEvent * event)
But I have to admit that I do not really understand the function. No idea how this should return a subclass of an QEvent...

[looooo]

mottosso/Qt.py#53 (comment) there is a difference in wrapping with shiboken and sip. We have to wrap first to a baseobject and then get the classname of the subclass and wrap again.

[InventorMentor]

Yes, I had already tried this suggested solution yesterday evening from the python side (using your example renderAreaCallback.py extended with wrapInstance ported to python 3 from
http://nathanhorne.com/pyqtpyside-wrap-instance/
buton win64 wrapInstance gives me the address overflow problem with the official shiboken.
However, a pure C++ solution would be much nicer, anyhow, linking against libshiboken, just relying on the current SWIG-part but addtionally converting converting the swig-pointer to PySide ones (not invoking the currently crashing PyEval_CallObject that apparently needs to be wrapped with gilstate ensure() / release according to some blogs [however, still crashed for me, presumably due to win64 address overflow], since when trying to wrap/unwrap in SoQt.i using Shiboken this call to reentering the python interpreter is not thread safe but it should be! It could be entirely avoided, however, on the C side in SoQt.i by calling Shiboken::createWrapper(qobject, /*python owns*/ false); and adding a compile time dependency to libshiboken; please cf.
http://lists.qt-project.org/pipermail/pyside/2012-September/000647.html

[looooo]

However, a pure C++ solution would be much nicer, anyhow, linking against libshiboken, just relying on the current SWIG-part but addtionally converting converting the swig-pointer to PySide ones...

I don't think relying on c++ shiboken library will simplify our lives. As the remaining issue here should be quite solveable, I will try to do so for linux. Maybe I have time to look at windows afterwards.

[InventorMentor]

Solveable, yes, definitely (either on the python [application] side using Nathan's function or much nicer transparently in SoQt.i). Correctness including thread safety first, i.e. also wrapping the reentrant calls to the python interpreter PyEval_CallObject in SoQt.i with gil ensure/release, however, this solution once it is working should add considerable and unnecessary run-time overhead, e.g. when generating many events by moving the mouse in SoQt and wrapping these events for PySide using run-time-only dependecies to shiboken and locking the gil each time and swapping threads going back an forth. The current SWIG "fallback" solution in SoQt.i already works fine on win64, only examples 9.1 (keyboards event pressing 'p' for printing) and 10.2 do not work with the current SWIG fallback and it is not possible to embed SoQtExaminerViewer in a PySide application which is the major issue from a pure python application development point of view. Just my two cents. Unfortunately, I will only find time again tomorrow afternoon at the earliest to get on to patching shiboken for win64. So long, all the best, Peter

[looooo]

Solveable, yes, definitely (either on the python [application] side using Nathan's function or much nicer transparently in SoQt.i)

the solution of Nathan still doesn't work for us. The problem is that a QEvent is not a QObject and therefor has no method to get it's subclass name. So somehow we have to get the subclass-name from the pure QEvent object. This not necessarily have to be done in python. I would simple call the corresponding python functions with the python-capi...

The current SWIG "fallback" solution in SoQt.i already works fine on win64, only examples 9.1 (keyboards event pressing 'p' for printing) and 10.2 do not work with the current SWIG fallback and it is not possible to embed SoQtExaminerViewer in a PySide application which is the major issue from a pure python application development point of view.

9.1 uses a SoEventCallback. I do not have insights in how the coin-callback-mechanism works, but example 10.2 could be simple solved by using also a coin.SoEventCallback. For me the absence of the possibility to set the callback directly is not a big problem.

in SoQt.i with gil ensure/release, however, this solution once it is working should add considerable and unnecessary run-time overhead

This is true for python callbacks anyway. But simple interactions are quite possible. eg: https://forum.freecadweb.org/viewtopic.php?f=22&t=21808#p169625

[looooo]

maybe it's also interesting how quarter translates the events to pivy/coin:
https://github.com/looooo/pivy/blob/master/pivy/quarter/devices/DeviceManager.py#L52L64

[looooo]

it's actually listed as a bug for shiboken:
https://bugreports.qt.io/browse/PYSIDE-31?jql=project%20%3D%20PYSIDE%20AND%20resolution%20%3D%20Unresolved%20ORDER%20BY%20assignee%20ASC%2C%20priority%20DESC

so I would say, we should use SoEventCallbacks for example 10.2 to workaround and make a note that QEvents are not handled properly with pyside.

[looooo]

on linux 10.2 works now. I also added a option to run the example with SoCallBack workaround:
python 10.2.setEventCB.py coin Maybe you can test this with windows.

[InventorMentor]

What an amazing contribution! Excellent work, well done! I reckon I still have to patch shiboken on win64, however, since I still receive the known address overflow problem testing example 10.2. Could you please fix just one minor issue specific to compilation with msvc14: in line 107 and 112 of SoQt.i the char* return type needs an additional const in order to be successfully compiled with msvc:
template <typename T> const char* get_typename(T& object)
I look forward to testing your contribution. Presumably I will only find enough time for that however, not before Thursday morning :-|

[InventorMentor]

Could you please commit a win64 specific modification to Inventor/misc/SoBase.i
unsigned long long __hash__() { return (unsigned long long) $self; }
otherwise $self - i.e. an SoBase object's 64-bit this pointer - is going to be truncated to 32-bit (cf. compiler warning below) when invoking the __hash__ function on win64. This shouldn't matter on linux64, since (unsigned) long there is already a 64 bit address. Unfortunately, there are still more such 64-bit addressing problems lurking around ... I'll post my current problems in a separate comment.

PS: The compiler warning that is going to be fixed by this is
pivy\coin_wrap.cpp(5466): warning C4311: 'type cast': pointer truncation from 'SoBase *' to 'long'

[InventorMentor]

To this end, I had no success with running example 10.2 in just fixing the 64-bit addressing problem in shiboken's 1.2.2 wrapInstance function as suggested by the Maya community. Therefore, I decided to also build PySide 1.2.4 with a patched shiboken 1.2.4 as described below resulting in even more address overflow problems with all of the examples :-( maybe FreeCAD makes use of a better implementation of PySide on win64?

Here is what I tried to this end in a more detailed description:

Following the discussion in
https://www.mail-archive.com/[email protected]&q=subject:%22Re%5C%3A+%5C%5BPySide%5C%5D+Shiboken+Large+Address+Bug%22&o=newest&f=1

and applying the fix suggested in
https://bugreports.qt.io/browse/PYSIDE-191

by

adding a line for a missing type declaration:
<primitive-type name="unsigned long long" />

and changing signature of wrapInstance to:
<add-function signature="wrapInstance(unsigned long long, PyType)" return-type="PyObject*">

this didn't fix the Overflow issue with example 10.2, however. Therefore, instead of using Chris Gohlke's prebuild win64 wheel for PySide 1.2.4, I decided to build one myself including the shiboken 64-bit address patch in wrapInstance above by

... cloning PySide from:
https://github.com/krrr/PySide-setup

... building it with:
C:\opt\libs\pyside-1.2.4>python.exe setup.py bdist_wheel --ignore-git --qmake=c:\Qt\4.8.7_msvc14\bin\qmake.exe --openssl="C:\Program Files\OpenSSL\bin"

(required some minor adjustments to setup.py for python 3.6)

... successfully built me a PySide wheel with the patched shiboken wrapInstance:
C:\opt\libs\pyside-1.2.4\dist\PySide-1.2.4-cp36-cp36m-win_amd64.whl

after its pip-installation to
python-3.6.1.amd64\Lib\site-packages\PySide

I had to copy the following three files shiboken.exe, shiboken-python3.6.dll, shiboken.pyd
one directory up directly into directory site-packages in order to directly support import shiboken in my python3 installation instead of from PySide import shiboken.

[InventorMentor]

Running python 10.2.setEventCB.py caused after 31 ms the following

Unhandled exception at 0x00000000748DAE6F (QtCore4.dll) in python.exe: 0xC0000005: Access violation reading location 0x00000000584A52E8.

raised in line 393 in file soqt-1.5.0\src\Inventor\Qt\SoQtComponent.cpp:
PRIVATE(this)->parent->installEventFilter(PRIVATE(this));

Callstack:
QtCore4.dll!00000000748dae6f() Unknown
soqt1.dll!SoQtComponent::SoQtComponent(QWidget * const parent, const char * const name, const int embed) Line 393 C++
soqt1.dll!SoQtGLWidget::SoQtGLWidget(QWidget * const parent, const char * const name, const int embed, const int glmodes, const int build) Line 179 C++
soqt1.dll!SoQtRenderArea::SoQtRenderArea(QWidget * parent, const char * name, int embed, int mouseInput, int keyboardInput) Line 887 C++
_soqt.cp36-win_amd64.pyd!00007ff99670eba1() Unknown

I receive the same exception now when trying to run all other SoQt examples.

[InventorMentor]

With trying python 02.1.HelloCone.pyw

OverflowError
TypeError: You need a shiboken-based type.
TypeError: You need a shiboken-based type.
TypeError: You need a shiboken-based type.

[looooo]

pyside for python3.5 is not really released. It works for linux, but needs some patching on windows. As far as I can tell, the conda version is working quite fine on windows. Maybe you can have a look at there recipe:
https://github.com/conda-forge/pyside-feedstock/blob/master/recipe/bld.bat
https://github.com/conda-forge/pyside-feedstock/blob/master/recipe/meta.yaml#L20

If we could setup a conda-recipe for soqt we could also build pivy with soqt on conda. Currently I only build coin for conda. This would allow to use pyside from conda...

[looooo]

I have successfully setup a conda-recipe for soqt on linux. On windows this doesn't work. Do you know how to compile soqt on windows?
recipe can be found here: https://github.com/looooo/FreeCAD_Conda/tree/master/soqt
bld.bat is the install-script for windows.

[InventorMentor]

Yes, I do, employing a msvc14 solution for python 3.6 that should also work with python 3.5 after some minor tweaks. I have also created a working msvc10 solution for python 3.4. Would this be good enough for you? Otherwise I could prepare a qmake project file or a corresponding CMakeList.txt if this would be of any help. To this end I have no experience with conda, yet.

N.B.: CPython versions on windows (and custom packages built for them) unfortunately depend on specific versions of the msvc runtime as documented here: https://wiki.python.org/moin/WindowsCompilers - msvc is available as a free community edition (requiring a shocking 8 gb of disk space) but at least it ships with the x64 compiler included by default. The (commercial) msvc10 pro also ships with an x64 compiler, however, the free msvc10 community edition doesn't / ships with a 32-bit x86 compiler only. Neverthless, there is a corrsponding freely available platform sdk available with x64 compiler support for msvc10. However,, I got the impression the sdk's C runtime was slightly different from CPython's 3.4 one which would lead to unexpectedruntime problems with custom build packes. Anyhow, welcome to the brave new windows world. I'm still shocked about their decision to make long on win64 an int32_t instead of an int64_t as was reasonably decided for linux64. This decision will cause a lot of crashes, trouble and demage when porting most well establised C code to more current 64 bit versions of windows :-(

It would be great if conda was hiding all these unnecessary build complexities for building CPython packages on windows ...

[looooo]

conda is quite a nice tool. It simplifies the build process. (At least it's my impression) We have already setup all dependencies of freecad as conda recipes... There are also very good documentation of conda out there. If you want I can give you some references....

Today I also tried the pivy binding of soqt with python3. (Previously I always used python2 to test the pyqt->pyside port) And there are still problem with 10.2.
So my roadmap is to:

1. get soqt bindings with pyside and python3.5 working on linux
2. get soqt compiled on windows
3. get soqt bindings with pyside and python3.5 working on windows

it would be nice if you could give me some instructions how to build soqt on windows. There is a cmake-branch available https://bitbucket.org/roboticslibrary/soqt/src, but there were some problems with the building. So the options are:

1. use/(fork) the cmake branch and repair the not working stuff. (actually I don't think it is much work) ->cmake should be straight forward to use on windows
2. use the tratitional install (.configure) (but I have no idea how to use it)

[looooo]

ok I know now how to run the ./configure in windows:
but the ./configure gives me now this errormessage:

configure: error: no acceptable ld found in $PATH

[InventorMentor]

Thanks for the information and that I may assist with your roadmap. Just out of my memory, configure relies on autotools and therefore on windows is only supported by the mingw compiler (already shipping with the WinPython distribution for example). Mingw has the advantage on windows that its build outputs do not depend on python version / msvc depended compiler runtimes but are self-contained executables and dlls in this respect. However, I'm unsure whether using it would be a good idea and I haven't done so on windows, since it does not seem to be the default way for building cpython packages on windows that depend on external shared libaries / dlls (cf. e.g. the well known python package repository for windows maintained by Chris Gohlke @ uci). Neverthless, some people manage to build dll-depended cpython packages for windows using mingw that are not specific to a particular cpython version. Therefore, I suggest that I transfer my current msvc14 solution to a qmake pro file (since soqt is for usage with qt anyhow) or modify the existing cmake-branch. It might become Saturday though before I will find enough time for that ...

[looooo]

with a minimal change soqt compiles for me on linux with cmake:
https://github.com/looooo/soqt/commit/548eacfcf82d2082480053d70c986042a269c3d6

but compiling the version I uploaded to github reports missing files. This is quite strange. Also creating a compressed file doesn't work. No idea whats wrong there.

[ 37%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtLightSliderSet.cpp.o
/home/lo/conda/conda-bld/soqt_1495181428356/work/soqt/src/Inventor/Qt/SoQtLightSliderSet.cpp:34:50: fatal error: Inventor/Qt/moc_SoQtLightSliderSet.icc: Datei oder Verzeichnis nicht gefunden
 #include <Inventor/Qt/moc_SoQtLightSliderSet.icc>

[looooo]

building directly from the source doesn't show any SoQtLightSliderSet.cpp

[  2%] Generating src/Inventor/Qt/moc_SoQtComponentP.icc
[  2%] Generating src/Inventor/Qt/moc_SoQtGLWidgetP.icc
[  4%] Generating src/Inventor/Qt/widgets/moc_SoQtThumbWheel.icc
[  5%] Generating src/Inventor/Qt/moc_SoQtMaterialList.icc
[  6%] Generating src/Inventor/Qt/moc_SoQtP.icc
/home/lo/git/soqt/src/Inventor/Qt/SoQtMaterialList.cpp:0: Note: No relevant classes found. No output generated.
[  8%] Generating src/Inventor/Qt/moc_SoQtMaterialSliderSet.icc
[  9%] Generating src/Inventor/Qt/moc_SoQtSliderSetBase.icc
[ 10%] Generating src/Inventor/Qt/moc_SoQtSignalThread.icc
[ 12%] Generating src/Inventor/Qt/moc_SoQtTransformSliderSet.icc
[ 16%] Generating src/Inventor/Qt/viewers/moc_SoQtFullViewerP.icc
[ 16%] Generating src/Inventor/Qt/viewers/moc_SoQtExaminerViewerP.icc
[ 16%] Generating src/Inventor/Qt/viewers/moc_SoQtPlaneViewerP.icc
[ 17%] Generating src/Inventor/Qt/widgets/moc_QtNativePopupMenu.icc
[ 19%] Generating src/Inventor/Qt/widgets/moc_SoQtGLArea.icc
Scanning dependencies of target SoQt
[ 20%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQt.cpp.o
[ 21%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtComponent.cpp.o
[ 23%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtGLWidget.cpp.o
[ 24%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtImageReader.cpp.o
[ 26%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtSignalThread.cpp.o
[ 27%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/6DOFEvents.cpp.o
[ 28%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtDevice.cpp.o
[ 30%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtInputFocus.cpp.o
[ 31%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtKeyboard.cpp.o
[ 32%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtMouse.cpp.o
[ 34%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/ExaminerViewer.cpp.o
[ 35%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/FullViewer.cpp.o
[ 36%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/PlaneViewer.cpp.o
[ 38%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/widgets/QtNativePopupMenu.cpp.o
[ 39%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/widgets/SoQtGLArea.cpp.o
[ 41%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/widgets/SoQtThumbWheel.cpp.o
[ 42%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoAny.cpp.o
[ 43%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtCommon.cpp.o
[ 45%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtComponentCommon.cpp.o
[ 46%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtCursor.cpp.o
[ 47%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtGLWidgetCommon.cpp.o
[ 49%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtObject.cpp.o
[ 50%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/SoQtRenderArea.cpp.o
[ 52%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtDeviceCommon.cpp.o
[ 53%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtInputFocusCommon.cpp.o
[ 54%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtKeyboardCommon.cpp.o
[ 56%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/SoQtMouseCommon.cpp.o
[ 57%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/devices/spwinput_x11.cpp.o
[ 58%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/editors/SoQtColorEditor.cpp.o
[ 60%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/editors/SoQtMaterialEditor.cpp.o
[ 61%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/engines/Engines.cpp.o
[ 63%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/engines/Format.cpp.o
[ 64%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/engines/RadioGroup.cpp.o
[ 65%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/ClickCounter.cpp.o
[ 67%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/ColorEditor.cpp.o
[ 68%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Frame.cpp.o
[ 69%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Image.cpp.o
[ 71%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Label.cpp.o
[ 72%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/MaterialEditor.cpp.o
[ 73%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Nodes.cpp.o
[ 75%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Pane.cpp.o
[ 76%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Position.cpp.o
[ 78%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/RadioButton.cpp.o
[ 79%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/SceneTexture2.cpp.o
[ 80%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Slider1.cpp.o
[ 82%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Slider2.cpp.o
[ 83%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/ToggleButton.cpp.o
[ 84%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/Translation.cpp.o
[ 86%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/ViewpointWrapper.cpp.o
[ 87%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/nodes/ViewportFix.cpp.o
[ 89%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/SoQtConstrainedViewer.cpp.o
[ 90%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/SoQtExaminerViewer.cpp.o
[ 91%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/SoQtFlyViewer.cpp.o
[ 93%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/SoQtFullViewer.cpp.o
[ 94%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/SoQtPlaneViewer.cpp.o
[ 95%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/viewers/SoQtViewer.cpp.o
[ 97%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/widgets/SoAnyThumbWheel.cpp.o
[ 98%] Building CXX object CMakeFiles/SoQt.dir/src/Inventor/Qt/widgets/SoQtPopupMenu.cpp.o

[InventorMentor]

The editor widgets for material properties and setting up a directional light were never ported to SoQt 1.5 and are not needed by most people, anyway, these days, since e.g. Qt includes now a very nice default widget for selecting colors (QRGBColorChooser if I remember correctly). Therefore, these legacy editors are also not part of linux packages of SoQt. You may want to just comment these out of the CMakeList ... (they are legacy from SoQt 1.2 and are based on Qt3 or even Qt2 and nobody saw a need to port them to Qt4. Once an SoQt-based ExaminerEmbed example will work with PySide the functionality of these legacy editors could very easily be replaced from the application side using QWidgets if needed.)

[InventorMentor]

I've decided to derive a qmake .pro file from my current msvc14 solution for SoQt-1.6.0a. Slowly, I'm getting there. It compiles fine now using nmake makefiles, however, I still get some linker errors. While porting the build system to qmake, I've found and fixed a win64 address trunctation problem in template file
soqt-1.6.0\src\Inventor\Qt\common\SoGuiComponentCommon.cpp.in
where unsigned long needs to be replaced multiple times with size_t for the lookup of QWidget addresses in a dictionary. However, this did not fix yet my problem with example 10.2 reported above. I'll continue to work on the linker problem tomorrow and hope to provide a win64-msvc compatible qmake pro file then. Best wishes

PS: The same fix needs to be applied in \soqt-1.6.0\src\Inventor\Qt\SoQtComponent.cpp

[InventorMentor]

The following qmake .pro successfully built me both soqt1.dll and soqt1.lib calling nmake from the "VS2015 x64 Native Tools Command Prompt":

# soqt1.pro
# tested with QMAKESPEC=win32-msvc2015 and QMake (2.01a) (Qt 4.8.7) on 64-bit Windows 10
# using x64 native compiler toolset applied with: %QTDIR%\bin\qmake soqt1.pro

TEMPLATE = lib
TARGET   = soqt1
QT      += gui opengl
CONFIG  += qt warn_on thread shared release
CONFIG  -= app_bundle uic

COINDIR       = /opt/libs/coin-4.0.0
SOQTDIR       = /opt/libs/soqt-1.6.0
SOQT_SRC      = $${SOQTDIR}/src
SOQT_MSVC_SRC = $${SOQTDIR}/build/msvc14/src

HEADERS = \
# Private Headers (excluded from build)
#  devices local includes
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoGuiDeviceP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoGuiInputFocusP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoGuiKeyboardP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoGuiMouseP.h \
$${SOQT_SRC}/Inventor/Qt/devices/SoQtDeviceP.h \
# $${SOQT_SRC}/Inventor/Qt/devices/SoQtLinuxJoystickP.h \
#  engines local includes
$${SOQT_MSVC_SRC}/Inventor/Qt/engines/SoGuiEngines.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/engines/SoGuiFormat.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/engines/SoGuiRadioGroup.h \
#  nodes local includes
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiClickCounter.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiFrame.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiImage.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiLabel.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiNodes.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiPane.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiPosition.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiRadioButton.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiSceneTexture2.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiSlider1.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiSlider2.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiToggleButton.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiTranslation.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiViewpointWrapper.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiViewportFix.h \
#  Qt local includes
$${SOQT_MSVC_SRC}/Inventor/Qt/SoGuiComponentP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoGuiGLWidgetP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoGuiP.h \
#$${SOQT_SRC}/Inventor/Qt/SoQtSignalThread.h \
#  src local includes
# $${SOQT_MSVC_SRC}/config.h \
$${SOQT_MSVC_SRC}/config-debug.h \
$${SOQT_MSVC_SRC}/config-release.h \
$${SOQT_SRC}/qt-config.h \
$${SOQT_MSVC_SRC}/soqtdefs.h \
#  viewers local includes
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoGuiExaminerViewerP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoGuiFullViewerP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoGuiPlaneViewerP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoGuiViewerP.h \
# Public Headers
#  Inventor/Qt headers
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQt.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtBasic.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtComponent.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtCursor.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtGLWidget.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtObject.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtRenderArea.h \
#  Inventor/Qt/devices headers
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtDevice.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtKeyboard.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtMouse.h \
#  Inventor/Qt/editors headers
$${SOQT_MSVC_SRC}/Inventor/Qt/editors/SoQtColorEditor.h \
#  Inventor/Qt/nodes headers
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiColorEditor.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SoGuiMaterialEditor.h \
#  Inventor/Qt/viewers headers
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtConstrainedViewer.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtExaminerViewer.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtFlyViewer.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtFullViewer.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtPlaneViewer.h \
#$${SOQT_SRC}/Inventor/Qt/viewers/SoQtPlaneViewerP.h \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtViewer.h \
#  Inventor/Qt/widgets headers
$${SOQT_MSVC_SRC}/Inventor/Qt/widgets/SoQtPopupMenu.h
# $${SOQT_SRC}/Inventor/Qt/widgets/QtNativePopupMenu.h \
# $${SOQT_SRC}/Inventor/Qt/widgets/SoQtGLArea.h \
# $${SOQT_SRC}/Inventor/Qt/widgets/SoQtThumbWheel.h

SOURCES = \
# Source Files
#  Inventor/Qt sources
$${SOQT_MSVC_SRC}/Inventor/Qt/SoAny.cpp \
$${SOQT_SRC}/Inventor/Qt/SoQt.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtCommon.cpp \
$${SOQT_SRC}/Inventor/Qt/SoQtComponent.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtComponentCommon.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtCursor.cpp \
$${SOQT_SRC}/Inventor/Qt/SoQtGLWidget.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtGLWidgetCommon.cpp \
$${SOQT_SRC}/Inventor/Qt/SoQtImageReader.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtObject.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/SoQtRenderArea.cpp \
$${SOQT_SRC}/Inventor/Qt/SoQtSignalThread.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtMouseCommon.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/spwinput_win32.c \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/spwinput_x11.cpp \
#  Qt/devices sources
$${SOQT_SRC}/Inventor/Qt/devices/6DOFEvents.cpp \
$${SOQT_SRC}/Inventor/Qt/devices/SoQtDevice.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtDeviceCommon.cpp \
$${SOQT_SRC}/Inventor/Qt/devices/SoQtInputFocus.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtInputFocusCommon.cpp \
$${SOQT_SRC}/Inventor/Qt/devices/SoQtKeyboard.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/devices/SoQtKeyboardCommon.cpp \
$${SOQT_SRC}/Inventor/Qt/devices/SoQtMouse.cpp \
#  Qt/editors sources
$${SOQT_MSVC_SRC}/Inventor/Qt/editors/SoQtColorEditor.cpp \
#  Qt/engines sources
$${SOQT_MSVC_SRC}/Inventor/Qt/engines/Engines.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/engines/Format.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/engines/RadioGroup.cpp \
#  Qt/nodes sources
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/ClickCounter.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/ColorEditor.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Frame.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Image.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Label.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/MaterialEditor.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Nodes.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Pane.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Position.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/RadioButton.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/SceneTexture2.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Slider1.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Slider2.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/ToggleButton.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/Translation.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/ViewpointWrapper.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/nodes/ViewportFix.cpp \
#  Qt/viewers sources
$${SOQT_SRC}/Inventor/Qt/viewers/ExaminerViewer.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtConstrainedViewer.cpp \
$${SOQT_SRC}/Inventor/Qt/viewers/FullViewer.cpp \
$${SOQT_SRC}/Inventor/Qt/viewers/PlaneViewer.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtExaminerViewer.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtFlyViewer.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtFullViewer.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtPlaneViewer.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/viewers/SoQtViewer.cpp \
#  Qt/widgets sources
$${SOQT_SRC}/Inventor/Qt/widgets/QtNativePopupMenu.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/widgets/SoAnyThumbWheel.cpp \
$${SOQT_SRC}/Inventor/Qt/widgets/SoQtGLArea.cpp \
$${SOQT_MSVC_SRC}/Inventor/Qt/widgets/SoQtPopupMenu.cpp \
$${SOQT_SRC}/Inventor/Qt/widgets/SoQtThumbWheel.cpp

win32 {

    DEFINES += COIN_DLL SOQT_INTERNAL SOQT_MAKE_DLL HAVE_CONFIG_H _WINDLL
    DEFINES -= UNICODE
    INCLUDEPATH += $${COINDIR}/include \
                   $${SOQT_MSVC_SRC} \
                   $${SOQT_SRC}

    Release {
        CONFIG += release
        DEFINES += NDEBUG _NDEBUG SOQT_DEBUG=0
        LIBS += $(COINDIR)/lib/coin4.lib
    }

    Debug {
        TARGET  = soqt1d
        CONFIG -= release
        CONFIG += debug
        DEFINES += DEBUG _DEBUG SOQT_DEBUG=1
        LIBS += $(COINDIR)/lib/coin4d.lib
    }

}

[InventorMentor]

on linux 10.2 works now. I also added a option to run the example with SoCallBack workaround:
python 10.2.setEventCB.py coin Maybe you can test this with windows.

Your workaround for 10.2 with option coin now "sort of" works for me under windows. However, I had to change line 74 in 10.2 from
intersection = (p0 + p1) / 2.0
to
intersection = (p0 + p1) * 0.5
otherwise, I get the following error when clicking the left mouse button in the render area:
TypeError: unsupported operand type(s) for /: 'SbVec3f' and 'float'
I've got no idea why this is so, since this division seems to be well defined in SbVec3f.i

[looooo]

sry for my late response. Thanks for the qmake file. But I fear my problem with soqt is related to some hg (mervcurial) suprepo stuff. I can build the project from a local clone of the soqt-repo. But I can't build it from a compressed file( which is the normal way to build conda-packages) So somehow I have to gather the sources in a final state, but no idea how to do that...

[InventorMentor]

I have not looked into "generating the final state" of the sources, but have been using the supplied msvc solutions. Visual Studio invokes some parameterized batch-files in a prebuild stage, subst'ing those "template" cpp.in files into final .cpp files but this might only be for wrapping the SoGui stuff. At a first glance, there does not seem to be a difference between these final state source files among visual studio versions. Therefore, I don't get to this end why there is a msvc version specific generation of these files, anyway. I'm going to be away tomorrow, but if it would be of any help to you I could upload my (size_t) patched soqt with final state msvc14 files into a dropbox with a download url (applogies for that, it would be just for supporting your integration into github). I'm going to be away tomorrow, but if you'll let me know, I could still do that today ...

[looooo]

i solved the first problem I have mentioned. It does compile now locally but not within the docker-container. somehow opengl isn't found properly:

opt/conda/conda-bld/soqt_1495876898985/work/soqt/src/Inventor/Qt/SoQtGLWidget.cpp:1308:3: error: ‘GLXContext’ was not declared in this scope
   GLXContext ctx = glXGetCurrentContext();

windows finishes the build process but has linking problems:

LINK : fatal error LNK1181: Eingabedatei "Coin.lib" kann nicht ge├Àffnet werden.
LINK failed. with 1181

[InventorMentor]

The first problem is a wrong code path due to a wrong or missing preprocessor definition, since glx - the binding of OpenGL to the X Window system is not relevant on windows.

The second poblem is due to a wrong or missing linker path to coin4.lib - the interface file to coin4.dll.

You could try to generate a Makefile using my qmake script above to modify and take over to conda its preprocessor defitions and linker options. Best regards and wishes and enjoy your weekend!

[looooo]

The first problem is a wrong code path due to a wrong or missing preprocessor definition, since glx - the binding of OpenGL to the X Window system is not relevant on windows.

the first problem is related to linux... and there only when building with docker.

You could try to generate a Makefile using my qmake script above to modify and take over to conda its preprocessor defitions and linker options. Best regards and wishes and enjoy your weekend!

I have never worked with qmake... Also I don't know which dependencies this would involve. Thats why I am still trying to make the cmake branch work...

[InventorMentor]

What about building on linux employing autotools configure as described e.g. here
https://github.com/iat-cener/tonatiuh/wiki/Installing-SoQt-For-Linux
and taking note and comparing the compiler and linker options?

To make our life easier, I presume we should focus for now on linux64 and win64, only, shouldn't we?

[looooo]

for linux this should be possible. good idea. so only the problem with coin.lib has to be fixed...

the conda packages are win64+py35 and linux64+py35 anyway. i will concentrate on py35 for some further time...

[InventorMentor]

Thanks and good to hear! Your CMakeLists.txt looks impressive to me. Unfortunately, things are not as tidy on windows as with linux' filesystem hierarchy standard where shared libraries live in a designated space. Coin3D pressumes on windows that the default location of dll interface .lib files of coin related libraries (such as coin4.lib itself, simage1.lib, quarter1.lib, soqt1.lib, simvoleon1.lib, dime, etc.) is %COINDIR%\lib. The corresponding dll's just have to live somewhere within the system's search path. Therefore, I reckon the issue with the linker not finding coin.lib is related to the implementation of find_package(Coin REQUIRED CONFIG). Also I could not spot any other (manual) settings of linker options in your CMakeLists.txt. Employing version 4.0.0a instead of the earlier 3.1.3 version of coin, the library name on windows has changed from coin3.lib to coin4.lib. I'm not aware of it having ever been coin.lib on windows, anyway. So, long story short, my suggestion is to

• verify where both coin4.lib and coin4.dll live on your windows box
• evolve the find_package cmake script for coin4 correspondingly
• alternatively, instead of using find_package commenting it out and setting a manual linker configuration to coin4.lib similarly to what I've done in my qmake script above

All the best and enjoy the sunny sunday :-)

[InventorMentor]

... it seems the relevant linker options that have to be adapted to coin4.lib live within sectionif(msvc) in the following file:
https://github.com/JenniferBuehler/inventor_viewer/blob/master/CMakeMacros/FindCoin.cmake

[looooo]

just compiling soqt and pivy...
there is this warning:

pivy/gui/soqt_wrap.cpp:4863:39: warning: comparison with string literal results in unspecified behaviour [-Waddress]
           if (get_typename(*event) == "QKeyEventEx")

I am not very familiar with C... Maybe you know the answer... :-)

linux soqt and pivy (py27) is uploaded. Next will be pivy (py35) and then switching to win...

ps.: I am now using the cmake-file to find coin3d from freecad:
https://github.com/FreeCAD/FreeCAD/blob/master/cMake/FindCoin3D.cmake

[looooo]

finally I have uploaded the soqt package. Conda now shoulöd have all necessary packages available for the the soqt bindings. so next is the testing of the bindings on linux python3.5.
https://anaconda.org/freecad/soqt/files

[InventorMentor]

Regarding the warning above, please cf.
https://stackoverflow.com/questions/2603039/warning-comparison-with-string-literals-results-in-unspecified-behaviour
You would like to compare strings not pointers to their starting addresses in memory. Therefore, I reckon it should rather be something like
if (strcmp(get_typename(*event), "QKeyEventEx") == 0)

[looooo]

thanks. Applied.
Just tested the windows conda package:
examples/soqt/renderAreaCallback.py has the same error as I have posted above.
examples/Mentor/10.2 has the known wrapping issue of the QEvents
examples/Mentor/10.2 with coin as argument doesn't work because of the division problem. I have applied your suggestion (*0.5 instead of / 2.0)
I think python3 __truediv__ should be used instead of __div__. I will update this once there is a FreeCAD release with python3. But at the moment I think it is not that important.

Btw: I did not code the cmake-file... This is a fork of soqt which can be found here:
https://bitbucket.org/roboticslibrary/soqt/overview

[InventorMentor]

Just to get a clearer picture ... there are soqt examples that actually work for you on win64, e.g. adapting 02.1.HelloWorld.py to soqt, aren't there?

With the examples that fail for you (after a few seconds when you are being asked to attach visual studio as a post mortem debugger), could you please confirm that the failing QObjects (subclasses of QWidget and QEvent) appear to have truncated addresses below 4 GB? Win64 is expected to assign all application-side addresses in a virtual address space above 4 Gb by default, however, my post mortem pointers are all smaller than address 0x00000000FFFFFFFF, i.e. all address bits above 32 bit were set to zero (corressponding to the (minimum) 8 leading zeros in hex address notation), presumeably due to some erroneous down cast(s) from uint64 (unsignd long long on both win32 and win64) to uint32 (unsigned long on windows being a uint64 on linux64 where it just scales transparently with size_t of the underlying cpu). Printing regular PySide widget addresses in win64 from python gives expected addresses > 4 GB, though (as can be seen e.g. in your first post in this thread).

[looooo]

Just to get a clearer picture ... there are soqt examples that actually work for you on win64, e.g. adapting 02.1.HelloWorld.py to soqt, aren't there?

I only tested the soqt examples. I belive 02.1 will also work with soqt. examples/soqt/minimal.py should be pretty similar...

But running 02.1 gives me your reported problem. (just tried it) But you are right, that is a very difficult one. I think I can't help with this.

[InventorMentor]

Thanks for confirming this issue. I have already found several wrong down-casts on win64 in PySide, Shiboken, and SoQt. A proper code review of all pointer casting code including Pivy would be a good thing and replacing all casts to long with casts to size_t (size of void pointer) which should work correctly on all platforms. I hope I can narrow down the problem further but obviously a systematic solution would feel better ;-)

[InventorMentor]

Thanks, I can confirm that __truediv__ fixes the division issue in example 10.2 wih python3.

Now, before diving too deep into the address overflow problem with PySide/SoQt under win64, I'd like to know whether the effort appears to be worthwile. Therefore, could you please confirm that examiner_embed4.py is actually working under py3-linux64 with pivy.gui.soqt employing PySide? This example needs the constructor call to be changed for py3 as reported in issue #19. If this example also does not work under linux64, I should eventually consider to migrate back to a 32-bit version of python 3 under win64 :-(

PS: Wasn't it Mr. Bill Gates himself, who asked famously "Who needs more than 640 kb of main memory, anyhow?" or to quote an anonymous source: "only bad programmers need a lot of memory / fast computers" ... so it seems I've to decide which camp to move to ;-)

[looooo]

just pushed the diff you have mentioned, plus a line which initialize the qApp. With this diff the examiner_embed4.py works for me with py2 and py3.

in windows there is reported an overflow error. But the application runs anyway.

this seems to be a known bug:
https://bugreports.qt.io/browse/PYSIDE-103

If we come up with a solution for that problem, I could add a patch to the conda-pyside-package.

ps.: now i see, you have already fixed this for your local shiboken :-)

[InventorMentor]

I really don't get this .. I've applied a fix to this bug already, however, you say for you this example is sort of working using your own PySide version under linux64 or even win64? (presumeably still without the fix) ... I've been using the PySide by krrr from here https://github.com/krrr/PySide as a baseline for msvc14 as used and recommended by Chris Gohlke but to this end without success ...

[looooo]

maybe you can try the conda packages on your machine. It's quite simple to work with conda. If you want I can give some instructions.

Also see here: conda-forge/pyside-feedstock#32 (comment)

[looooo]

just uploaded a pyside package: https://anaconda.org/freecad/pyside/files
this version has your reported problem fixed. The overflow-error has gone. But 10.2 is still not working.

[looooo]

the quarter problems are were related to this function:
looooo@79c1686

https://srinikom.github.io/pyside-docs/PySide/QtCore/QThread.html#PySide.QtCore.PySide.QtCore.QThread.currentThreadId

[InventorMentor]

Many thanks! First of all it is very encouraging to folllow all your recent progress providing soqt also for pivy-win64-py3. From an Inventor Mentor point of view, once example 10.2 will work and an SoQtExaminerViewer is embeddable into a pyside application, you could declare victory and bump pivy's version number to 1.0 :-) but maybe this is just me dreaming of a not too distant future ...

I'm currently using winpython 3.6, therefore, I could not apply your prebuild pyside package above. Also following the discussion in conda-forge/pyside-feedstock#32 size_t is obviously preferable to unsigned long long since it will preserve 32-bit compatibility. I still don't get why applying the same fix to https://github.com/krrr/PySide doesn't solve the wrapInstance issue for me. Could you please point me to your pyside sources so that I can use them to build a patched pyside-win64-py36 myself?

[looooo]

thanks very much. I am pretty sure once we have solved the 10.2 problem we will find other problems too. But I would like to release a 0.6.2 version, and therfore at least some issues should be solved.

regarding the source: it's all inside the meta.yaml file: https://github.com/looooo/pyside-feedstock/blob/master/recipe/meta.yaml#L9

best is to:

1. clone the recipe
2. download the source
3. extract the source
4. git init
5. git apply <pathes>

at least conda does these steps automatically...

[InventorMentor]

Many thanks, I'm going to give this recipe a try tomorrow. Having all functionality-wise relevant inventor mentor examples working now (at least with soqt, eventually also even with pivy-quarter, many thanks for your recent commit, I'll definitely check this out tomorrow) appears to be a very good baseline for a pivy release (feature completeness demonstrated on both py2/py3 and linux/windows 32/64.

Just one more question out of my memory: have you already fixed SoBase.i with size_t as described in one of my posts above and also I think that the original soqt 1.6.0a sources need to be patched correspindingly to wrapInstance in a few places. If you grep for unsigned long or maybe better just for long, you will find in two or three files (SoQtComponent.*, etc) down-casts of memory addresses used in a dictionary for QWidget/QEvent lookup internally in SoQt. This is expected to fail on win64, so again replacing these casts wih ones to size_t instead should be the correct and portable thing to do. I can look up the details of my changes to soqt tomorow if you want. hopefully, this will finally close the remaining issue with 10.2 ...

[looooo]

found one in SoBase:
https://github.com/looooo/pivy/blob/636ddbd3a47b8aef9a6277643287cf2e97c14156/Inventor/misc/SoBase.i#L7

but the other files are coin related...
I just started a poll at freecad to fork coin, So maybe we can patch this there... :-)
https://forum.freecadweb.org/viewtopic.php?f=10&t=22714

[looooo]

I think the remaining problem of 10.2 is related to my implementation of get_typename:
https://github.com/looooo/pivy/blob/master/interfaces/soqt.i#L114
This returns for example:
class QMouseEvent
but actually it should return only
QMouseEvent
So somehow the first 6 characters should be removed. I tried with std::string and string.erase but wasn't successfully. Maybe you have got an idea how to remove the first characters.

[InventorMentor]

I'm not really convinced that character string operations are the most efficient thing to do when handling events, though modifying pointers to the starting character of a string should be very efficient (however, for safety concerns, this introduces assumptions of non-unicode C-style strings always being prefixed with string "class "; the first assumption [1 byte/char strings] should be pretty safe since the signature of the function is dealing with char*, anyway; however, I'm not so sure about the second assumption being always met; could there ever occur empty strings for example? --> then a crash or undefined behaviour would result); if both assumptions are always met, my suggestion is to use something like in the second line of the following for returning the result (just be adding +6 in the return statement):

const char* simpleNonUnicodeAsciiCStyleString = "class QMouseEvent";
const char* shortenedVersion = simpleNonUnicodeAsciiCStyleString + 6;  // 6 == sizeof("class "), 1 byte/char
printf("shortened version= \"%s\"", shortenedVersion); // trimmed from left by six bytes (ascii characters)

[looooo]

I knew you will have a solution for this : ) Just tried it and it works quite well.
I have gone through all Mentor examples on windows and made a list of the not working ones: https://github.com/looooo/pivy/issues/20

If you can reproduce the working 10.2 example I think we can close this issue : )

[looooo]

and there is a new one!
testing py36-> https://travis-ci.org/looooo/pivy/jobs/237436673#L665

[InventorMentor]

May Betrand Meyer (of "programming by contract" fame, i.e. explicit contracts between caller and callee, explict specification of pre- and postconditions for subroutines) forgive us for just making the second assumption, but at least we could put the burden on the caller by makig these assumptions explicit in a comment (taking the role of such a contract :)

[InventorMentor]

With fixing 10.2, I'm unfortunately still not there yet. I've built PySide according to your recipe above (and had to make some destructors public for that in Qt-4.8.7 on the way), but I still receive a truncated 32-bit this-pointer in line 392 of file soqt-1.6.0\src\Inventor\Qt\SoQtComponent.cpp

PRIVATE(this)->parent->installEventFilter(PRIVATE(this));

I still get the folllowing:
Unhandled exception at 0x000000006C61ADFF (QtCore4.dll) in python.exe: 0xC0000005: Access violation reading location 0x0000000073969D58.
Callstack:

soqt1.dll!SoQtComponent::SoQtComponent(QWidget * const parent, const char * const name, const int embed) Line 393	C++
soqt1.dll!SoQtGLWidget::SoQtGLWidget(QWidget * const parent, const char * const name, const int embed, const int glmodes, const int build) Line 179	C++
soqt1.dll!SoQtRenderArea::SoQtRenderArea(QWidget * parent, const char * name, int embed, int mouseInput, int keyboardInput) Line 887	C++

PS:
In file SoQtComponent.cpp I now cast to SbDictKeyType instead of size_t (originally just long truncating QObject*) which reports me on win64 when calling
sizeof(SbDictKeyType) = 8 bytes with SbDictKeyType being defined in coin3d's Inventor/SbDict.h as
typedef uintptr_t SbDictKeyType; // aren't uint64_t-pointers memory aligned? I'm not sure whether this is also the case with void* or in this case QObject*, respectively.

My changed typecasts in SoQtcomponent.cpp are now
line 144: SbBool b = SoQtComponentP::cursordict->find((SbDictKeyType)cc, qc);
line 191: SoQtComponent.cpp: SoQtComponentP::cursordict->enter((SbDictKeyType)cc, c);
but still get the same problem above with 10.2:

[looooo]

I can confirm the problem still exists in win8 and win10. 10.2 and quarter-examples work on win7 (at least for me).
on win8 at least the quarterWidget was working... On windows 10 it seems quarter does not work either. Different windows versions seems to be much more worse then different linux distros... ; )

[InventorMentor]

It's sort of interesting that 10.2 actually works for you under win7. Could you please give furher details about your working configuration, e.g, such as ...
Win7 64 bit with 8 GB RAM
WinPython 3.6.1 - 64 bit
SoQt 1.6.0a with or without patch ... build with recipe using msvc14
Pivy 0.6.2 branch master, commit ....
Coin3d 4.0.0a with/without patch .... using msvc14 shared release build
Qt 4.8.7 with/without patch ... msvc 14 build
PySide 1.2.4 build using recipe ...

I reckon that the win64 issue is with pivy's SoQt binding and/or eventually also with SoQt itself. My shiboken built with your PySide recipe gives me now valid 64 bit cppPointers for QObjects created with PySide under 64 bit win10. Maybe win7 64 bit allocates some application addresses still below 4 GB so that a pointer truncation to 32-bit does not necessarily crash the application while 64 bit win8 / win10 assign application pointers to virtual addresses greater than 32 bit (4 GB), while 32-bit addresses below 4 GB are reserved for the operating system itself (this used to be 2 GB only by default in 32 bit Windows XP but could be changed via a registry setting to 1 GB, so applications had a maximum of 2 or 3 GB available) but I'm just speculating and still have to research that.

PS: Can you actually import shiboken in your working win7 configuration or is your soqt binding there making use of its swig-based object wrapping fallback instead?

[looooo]

win7 64 bit 16gb ram
for version info, source,... please have a look at the build recipes which can be found in the FreeCAD_Conda repository: https://github.com/looooo/FreeCAD_Conda Basically the sources are taken from the pivy release. soimage, soqt are slightly changed. Coin is patched in the recipe.

In my testenvironment I have these packages:

boost-cpp                 1.64.0                   vc14_0  [vc14]  conda-forge
ca-certificates           2017.4.17                     0    conda-forge
certifi                   2017.4.17                py35_0    conda-forge
coin3d                    4.0.0                    vc14_4  [vc14]  freecad
icu                       58.1                     vc14_1  [vc14]  conda-forge
jpeg                      9b                       vc14_0  [vc14]  conda-forge
libiconv                  1.14                     vc14_4  [vc14]  conda-forge
libpng                    1.6.28                   vc14_0  [vc14]  conda-forge
libtiff                   4.0.6                    vc14_7  [vc14]  conda-forge
libxml2                   2.9.4                    vc14_4  [vc14]  conda-forge
libxslt                   1.1.29                   vc14_4  [vc14]  conda-forge
openssl                   1.0.2h                   vc14_3  [vc14]  conda-forge
pip                       9.0.1                    py35_0    conda-forge
pivy                      0.6.2                py35_dev_3  [vc14]  freecad
pyside                    1.2.4               py35_vc14_6  [vc14]  freecad
python                    3.5.3                         3    conda-forge
qt                        4.8.7                    vc14_7  [vc14]  conda-forge
setuptools                33.1.1                   py35_0    conda-forge
simage                    1.7.0                    vc14_0  [vc14]  freecad
soqt                      1.5.0                    vc14_0  [vc14]  freecad
vc                        14                            0    conda-forge
vs2015_runtime            14.0.25420                    0    conda-forge
wheel                     0.29.0                   py35_0    conda-forge
wincertstore              0.2                      py35_0    conda-forge
zlib                      1.2.11                   vc14_0  [vc14]  conda-forge

I tested the exactly same repository on win8 and win 10.

To get the same environment on another machine you can do the following:

• download miniconda and install it: https://conda.io/miniconda.html
• start anaconda terminal and type:
  conda config --add channels conda-forge
conda config --add channels freecad
• create environment
  conda create -n pivy_35 pivy pyside python=3.5
  after installation is ready activate environment
  activate pivy_35
• cd into pivy-dir and run examples.

[InventorMentor]

Thanks, I will do that. I've got access to a 64 bit win7 machine, too. If it works my plan is to replace various modules in the hope that one of these will produce the bug again to narrow down the culprit. Before I do that could you please verify that you can directly import shiboken on this win7 installation? The swig-fallback in soqt seems to work much better across windows versions for most examples than the shiboken-based default wrapper of soqt ...

[looooo]

import shiboken works for:

• win7 64 python3.5.3
• win7 64 python3.6.1
• win10 64 python3.6.1
• win10 64 python3.5.3
  I belive the same is true for win8.

actually I made some mistakes previously on windows10. Quarter does work for me. Only 10.2 without coin argument crashes. (Maybe I have installed an older version...)
Also running 10.2 with the coin argunment prints a Overflowerror, which win7 does not.

OverflowError
TypeError: You need a shiboken-based type.
TypeError: You need a shiboken-based type.
TypeError: You need a shiboken-based type.

So maybe again some problems with pyside.

[looooo]

some further observation in win10:

• starting soqt/minimal.py several times will show the viewer sometimes.
• it's definetly a problem with soqt.
• maybe a wrapping issue. But I don't know where it is comming from.

[InventorMentor]

On 64-bit win10, example 10.2 only works as expected for me with option coin when using the swig-based soqt-binding (without an installed shiboken). Without option coin and without shiboken installed manually to Lib\site-packages (to disable the direct import shiboken expected by pivy instead of from PySide import shiboken as provided by the default PySide installation) I receive an

AttributeError: 'QEvent' object has no attribute 'type'
Traceback (most recent call last):
  File "10.2.setEventCB.py", line 140, in myAppEventHandlerQt4
    if anyevent.type() == QEvent.MouseButtonPress:

On 64-bit win10, with shiboken installed manually to Lib\site-packages running example 10.2
gives me with option coin random startup crashes of the application due to 32-bit pointer truncation as reported above and only sometimes when I start 10.2 the render area actually appears but then I get the OverflowError / shiboken-based type error message you reported above. Otherwise, the example is usable when it actually made it to startup. Omitting option coin [as the expected default use case for example 10.2 with shiboken manually installed to Lib\site-packages], however, starts up only occassionaly (randomly) but then I'm unable to draw in the render area and it prints the error message

AttributeError: 'QEvent' object has no attribute 'type'
OverflowError
Traceback (most recent call last):
  File "10.2.setEventCB.py", line 140, in myAppEventHandlerQt4
    if anyevent.type() == QEvent.MouseButtonPress:

In random cases, when example 10.2 does not start up at all with shiboken and crashes directly on startup for me, then the attached post-mortem debugger halted at an exception in this constructor call

SoQtGLWidget::SoQtGLWidget(QWidget * const parent,
                           const char * const name,
                           const SbBool embed,
                           const int glmodes,
                           const SbBool build)
  : inherited(parent, name, embed),
    waitForExpose(TRUE),
    drawToFrontBuffer(FALSE)

however, then the pointers like parent can not be inspected from the debugger with message Variable is optimized away and not available.

PS: Since under 64-bit win10 example 10.2 with option coin apparently always works as expected when import shikoken is not available (just with the swig-based 'fallback' wrapping of pivy.gui.soqt) , I reckon we may exclude libraries Coin3D and SoQt as the source of this issue. Testing a corresponding native C++ example 10.2 implementing a C++ version of myAppEventHandlerQt4(myRenderArea, anyevent) also works reliably and as expected on 64-bit win10, providing further evidence that this issue is not with native Coin3D 4.0.0a, SoQt 1.6.0a, or Qt-4.8.7, but must be with pivy, its's shiboken-based soqt-binding, pyside, or shiboken.

[looooo]

just came across these two warnings:

pivy\coin_wrap.cpp(5467): warning C4311: 'type cast': pointer truncation from 'SoBase *' to 'long'
pivy\coin_wrap.cpp(5467): warning C4302: 'type cast': truncation from 'SoBase *' to 'long'

[InventorMentor]

Test case: running failing example 10.2 on 64-bit win10 with available import shiboken multiple times until it randomly proceeds to show the render area after multiple attempts; The OverflowError reported above is printed in the body of the following failing if statement attempting to call shikobken_wrapinst_func in line 43 of file pivy/Inventor/Qt/SoQtRenderArea.i:

if (!(qev = PyEval_CallObject(shiboken_wrapinst_func, arglist))) {
            PyErr_Print();  // prints "OverflowError"
}

[looooo]

so this is again a problem with the pointer-size? Allthough it is now set to size_t.
regarding this site: https://msdn.microsoft.com/en-us/library/323b6b3k.aspx
size_t can be unsigned __int64 or unsigned integer, depending on the target platform...

[looooo]

unsigned long long has the same problem. If I understand it correctly:
If the wrapping fails (OverflowError) the example works because for the minimal example no pyside type is necesarry. If the wrapping does work, there is a crash (because of wrongly wrapped qt-object)

[InventorMentor]

Working backwards from line 43 in SoQrarenderArea.i, I'll have to insert printf statements to see what is going on and maybe to test assumptions and spot differences against a working scenario on win7 or eventually linux64, since I do not have a debug build of py3, pivy, qt, ... you name it .... therefore, the good old printf debugger should be my best bet ... some Maya folks apparently had a very similar OverflowError: need a shiboken based type ... and recommend as a solution to import shiboken only as from PySide import shiboken - the printf debugger will not be too easy to implement, however, because of the convoluted PyObject structures involved. Maybe you can verify that shiboken gets imorted correctly into pivy on win10 ...

[cgohlke]

... this didn't fix the Overflow issue with example 10.2, however. Therefore, instead of using Chris Gohlke's prebuild win64 wheel for PySide 1.2.4, I decided to build one myself including the shiboken 64-bit address patch in wrapInstance above by ...

FWIW, I have updated my PySide wheels with the shiboken 64-bit address patch. The OverflowError should be fixed.

[InventorMentor]

Dear Chris Gohlke, thank you very much indeed, I'll be most happy to try this. I can confirm that the issue for me on win10 was shiboken.wrapInstance
as I can now confirm on my machine using a minimal test case taken from https://bugreports.qt.io/browse/PYSIDE-103

from PySide import QtCore, shiboken
shiboken.wrapInstance(0xFFFFFFFF, QtCore.QObject)  # Overflow error --> largest 32-bit address, howevsser, this should definitely work on all systems
shiboken.wrapInstance(0x100000000, QtCore.QObject) # Overflow error --> an actual 64-bit address > 4 GB (the first non 32-bit address)

However, a bit surprisingly, a slightly smaller 32-bit address
shiboken.wrapInstance(0xFFFFFFF, QtCore.QObject)
works for me on 64 bit win10.

Now I can test this against your wheel .... I reckon you made my day :-)

[cgohlke]

Works for me:

Python 3.6.1 (v3.6.1:69c0db5, Mar 21 2017, 18:41:36) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from PySide import QtCore, shiboken
>>> shiboken.wrapInstance(0xFFFFFFFF, QtCore.QObject)
<PySide.QtCore.QObject object at 0x00000143E4F58288>
>>> shiboken.wrapInstance(0x100000000, QtCore.QObject)
<PySide.QtCore.QObject object at 0x00000143E4F583C8>

[InventorMentor]

finally, this minimal test case above also works for me, too :-) Thank you very much indeed! ... now, I'm looking forward to test example 10.2 and examinerEmbed4.py (testing the embedding of a Coin3D / OpenInventor examiner viewer into a PySide application that also failed on 64-bit win10 to this end)

[InventorMentor]

Unfortunately, running example 10.2 still fails for me with an
Unhandled exception at 0x00000000541AADFF (QtCore4.dll) in python.exe
coming from a 32-bit address on my 64-bit win10 machine using 64-bit WinPython 3.6.1.
QtCore.dll is on my system a self-compiled 64-bit msvc14 release dll and lives here (also in the system's search path):
C:\Qt\4.8.7_msvc14\bin\QtCore4.dll

[InventorMentor]

after recompiling pivy, I interestingly receive now a different error message than before:
OverflowError: can't convert negative int to unsigned

[InventorMentor]

@looooo you may want to try and add the following script to pivy/tests:

# shiboken_64bit_test.py
from PySide import QtCore
import shiboken  # as expected by pivy (instead of the default "from PySide import shiboken")

q = QtCore.QObject()
ptr = shiboken.getCppPointer(q)
print("CppPointer to an instance of PySide.QtCore.QObject = 0x%016X" % ptr[0])

# None of the following is expected to raise an OverflowError on 64-bit systems
shiboken.wrapInstance(0xFFFFFFFF, QtCore.QObject)  # largest 32-bit address
shiboken.wrapInstance(0xFFFFFFF, QtCore.QObject) # a regular, slightly smaller 32-bit address
shiboken.wrapInstance(0x100000000, QtCore.QObject) # an actual 64-bit address (> 4 GB, the first non 32-bit address)
shiboken.wrapInstance(0xFFFFFFFFFFFFFFFF, QtCore.QObject) # largest 64-bit address

[looooo]

added your test.

just a side note:

import shiboken # as expected by pivy (instead of the default "from PySide import shiboken")

I think shiboken should be importable directly. At least that was my conclusion from reporting it to conda:
conda-forge/pyside-feedstock#31

the test fails for win10 and ubuntu at:

ERROR: testAdresses (__main__.ShibokenTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "coin_tests.py", line 1226, in testAdresses
    shiboken.wrapInstance(0xFFFFFFFF, QtCore.QObject)  # largest 32-bit address
OverflowError

ps.: same is also true for win7. does this work for you?

[InventorMentor]

Yes, it works for me, but I don't know why. It should have worked with the patch for PySide-103, however, it didn't ... both my patched build based on krrr's PySide and according to your PySide recipe failed for me regarding shiboken wrapInstance. However, Chris Gohlke, suddenly dropped by to the rescue yesterday and provided a python wheel with a PySide that ships with a working 64-bit shiboken for windows ... but there is no source code and I can live without it for my needs .... I reckon FreeCAD's PySide, however, will also need this fix... apparently Autodesk is also publishing a fixed PySide on its website for Maya 2015.

PS: I've also found the bug :-) example 10.2 and examinerEmbed4 now happily work for me on 64-bit win10 ... I still have to investigate things a bit further and need to tidy up ...

[InventorMentor]

btw ... the bug was a type downcast error from a 64-bit address (of a void*) to a (signed 32-bit) int instead of the needed and plattform-independently correct size_t. There may be more such bugs lurking around ... I'll check that and will post a fix within the next couple of days!

[looooo]

wow, that are very good news. thanks for all the work!

[InventorMentor]

Sorry for my late reply. I needed to take a little break away from pivy after diving deep into this issue. Please find my patched versions of pivy/Inventor/Qt/SoQtRenderArea.i
https://www.dropbox.com/s/6v0mmvk3638vdq8/SoQtRenderArea.i?dl=0
and pivy/interfaces/soqt.i
https://www.dropbox.com/s/i2eydxzeixctc80/soqt.i?dl=0
ready for you to download under the above two URLs. Please copy their file contents over their existing versions in your current git clone on branch master and please review a git diff of my changes and check whether you are happy with them and would agree to keep them. All relevant examples including 10.2 (except just examples 9.1 and 12.4, but I'll write seperate issues for these later) now work happily as expected for me under 64-bit Windows 10. Please note that the complete fix to this issue also requires a working 64-bit version of shiboken for windows (e.g. as provided by Chris Gohlke's recent PySide wheels).

I feel it would be really good if we knew how the build these patched wheels ourselves just in case we need to fix more issues in PySide. Also, please note that I've not patched the python2 version pivy/interfaces/soqt2.i, yet. This still needs to be done. My posted fix should also work employing python 3 with win32, linux32 and linux64, however, I couldn't test this. Maybe you have the means to do that. If you are happy with the proposed solution and find it is working also on linux then I'd suggest that you may finally want to close this issue :-)

[looooo]

thanks for your collaboration.
can you give me your e-mail address so I can add you as a contributor.

[looooo]

the pyside problem was patched in the conda-package: conda-forge/pyside-feedstock@bd60246

I think I will try to create pivy-packages for win and linux this weekend and look if the problems on win10 is still there.

@cgohlke do you have a list of patches you use for your pyside-package?

[cgohlke]

@cgohlke do you have a list of patches you use for your pyside-package?

Source code is at http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyside in PySide-1.2.4-vc14-x64.zip.
To fix the OverflowError, see the changes in conversions.h and sbkconverter_p.h.

[looooo]

thanks a lot @cgohlke . The diff of these files can be found here: https://github.com/looooo/pyside-feedstock/blob/master/recipe/win_pointer_conversation_fix.patch

[looooo]

finally this solves 10.2 on windows 10 and conda. That is a big step forward :-)
so what is missing:

• add InventorMentor as contributer of looooo@eee5527
• apply this commit to python2 (looooo@5e1dc41)
• move pyside test in another file (there is still one test failing on win7, can we remove this one?)

[looooo]

@InventorMentor I can add you as author of the mentioned commit later anyway. Just tell me how ou would like to handle this.
I close this issue now as I think this example should work. At least I have tested with conda on:
linux py34, py35, py36
win7, win8, win10