Giter Site home page Giter Site logo

mkl dlls and mangling about delvewheel HOT 6 CLOSED

adang1345 avatar adang1345 commented on August 22, 2024
mkl dlls and mangling

from delvewheel.

Comments (6)

adang1345 avatar adang1345 commented on August 22, 2024

@samuelstjean Can you upload and attach the wheel that you tried repairing? I want to investigate further why the --no-mangle flag is not working.

from delvewheel.

samuelstjean avatar samuelstjean commented on August 22, 2024

I tried a few other things in the meantime, just need to find back the right file I guess.

from delvewheel.

samuelstjean avatar samuelstjean commented on August 22, 2024

spams_bin-2.6.3-cp37-cp37m-win_amd64.zip
This one should do it, looks like if your module loads A.dll, which then loads B.dll, you need to specify manually B.dll to be added. Not mangling A.dll as suggested seems to means it is untouched, and still tries to load B.dll, even if it will get renamed in the process later on.
Now add B.dll wants to load a bunch of other stuff (here probably the mkl_core.1.dll file) and it will just break on runtime.

And this is how I used it

(openblas) C:\Users\Samuel\Desktop\spams-python-patch2>delvewheel repair spams_bin-2.6.3-cp37-cp37m-win_amd64.whl --add-path c:\cibw\intelmkl.redist.win-x64.2021.3.0.524\runtimes\win-x64\native --add-dll libiomp5md.dll;mkl_avx.1.dll;mkl_avx2.1.dll;mkl_avx512.1.dll;mkl_core.1.dll;mkl_def.1.dll;mkl_intel_thread.1.dll;mkl_mc.1.dll;mkl_mc3.1.dll;mkl_rt.1.dll;mkl_sequential.1.dll;mkl_tbb_thread.1.dll;mkl_vml_avx.1.dll;mkl_vml_avx2.1.dll;mkl_vml_avx512.1.dll;mkl_vml_cmpt.1.dll;mkl_vml_def.1.dll;mkl_vml_mc.1.dll;mkl_vml_mc2.1.dll;mkl_vml_mc3.1.dll --no-mangle-all
repairing spams_bin-2.6.3-cp37-cp37m-win_amd64.whl
finding DLL dependencies
copying DLLs into spams_bin.libs
skip mangling DLL names
calculating DLL load order
patching spams\__init__.py
patching spams_wrap\__init__.py
updating spams_bin-2.6.3.dist-info\RECORD
repackaging wheel
fixed wheel written to C:\Users\Samuel\Desktop\spams-python-patch2\wheelhouse\spams_bin-2.6.3-cp37-cp37m-win_amd64.whl

It's pretty much shoving in all the dll included from the mkl, not mangling them somehow silently fails later on, but that's likely something different or another missing dll.

from delvewheel.

adang1345 avatar adang1345 commented on August 22, 2024

Before we can talk about whether name-mangling is causing an issue here, there is a bigger concern to address.

delvewheel is best suited for the situation where the DLLs are loaded via the DLL import table. In this situation, if a file A.dll depends on B.dll, then B.dll is loaded automatically at the time that A.dll is loaded. If the DLL dependencies are loaded at runtime using LoadLibraryEx() from C/C++, then delvewheel cannot determine the dependencies automatically.

It looks like the wheel you're trying to repair falls into the second case. Looking at the wheel you uploaded, I see that _spams_wrap.cp37-win_amd64.pyd depends on vcruntime140_1.dll, mkl_rt.1.dll, msvcp140.dll, and vcomp140.dll, based on the DLL import tables. Given that mkl_rt.1.dll has no other dependencies (other than Windows system libraries) listed in its import table, I suspect that either mkl_rt.1.dll or _spams_wrap.cp37-win_amd64.pyd loads its dependencies using the LoadLibraryEx() function. This situation makes wheel repair a lot more difficult. I include the --add-dll flag to try to handle this situation, but this is not a complete solution. For example, mkl_rt.1.dll or _spams_wrap.cp37-win_amd64.pyd might be passing certain flags to LoadLibraryEx() that affect the location where DLL dependencies are searched, and I have not tested how the different flags may affect the way that delvewheel-repaired wheels behave.

The first thing to determine would be to check whether all DLL dependencies are being included in the wheel. You'll have to check this yourself because (presumably) dependencies are loaded at runtime using LoadLibraryEx(). If you are still encountering errors, then I can try to help. In that case, you should tell me exactly what error occurs and how to reproduce the error.

from delvewheel.

samuelstjean avatar samuelstjean commented on August 22, 2024

I see, it's probably the mkl_rt.dll, its purpose is to provide only one lib to link to, and then it loads a bunch of other dll at runtime depending on what your processor supports. I'll try to link it traditionally to see if it has more success, but it does seems like a weird corner case since stuff is loaded dynamically I guess.

from delvewheel.

samuelstjean avatar samuelstjean commented on August 22, 2024

Fixed it by using a regular dll, but I guess from my initial question it probably falls outside the scope of the project since dependencies of dependencies is tricky.

from delvewheel.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.