Failed to load native whisper library.

[sandrohanea]

Failed to load native whisper library.

[sandrohanea]

Hello @MikePittAge ,
Unfortunately, that is breaking the functionality on Win11 LoadLibraryEx returns non-empty result but the library cannot be used and some AccessViolationException is thrown (additionally, according to https://learn.microsoft.com/en-us/windows/win32/api/libloaderapi/nf-libloaderapi-loadlibraryexa it's not recommended)

However, just pushed a commit to expose the ILibraryLoader so you will be able to do something like this:

internal class MyCustomWindowsLibraryLoader : ILibraryLoader
{
    public LoadResult OpenLibrary(string filename)
    {
        var loadedLib = LoadLibraryEx(filename, 0x00000001);

        if (loadedLib == IntPtr.Zero)
        {
            var errorCode = Marshal.GetLastWin32Error();
            var errorMessage = new Win32Exception(errorCode).Message;
            return LoadResult.Failure(errorMessage);
        }

        return LoadResult.Success;
    }

    [DllImport("kernel32", SetLastError = true, CharSet = CharSet.Auto)]
    private static extern IntPtr LoadLibraryEx([MarshalAs(UnmanagedType.LPTStr)] string lpFileName, IntPtr hFile, uint dwFlags);

}

And then, before creating any WhisperFactory, you can do:

NativeLibraryLoader.SetLibraryLoader(new MyCustomWindowsLibraryLoader ());

using var factory = WhisperFactory.FromPath(opt.ModelName);

var builder = factory.CreateBuilder()
        .WithLanguage(opt.Language);
...

[GewoonJaap]

Seems to work on 64bit, but not on ARM?

[sandrohanea]

Hello,
Indeed, we don't have yet support for ARM, but this will be added soon. Will keep you posted.

[GewoonJaap]

Cool! I have been trying to play around with whisper.cpp to get the DLL working (made a fork), sadly it doesn't output the right text as of now, so will try and get that fixed, but don't know how haha. It only outputs comma's. But the DLL compiles so that's something

[adamnova]

When you get weird texts like commas or some random repeating text it might be an issue with the data you supply to the library, like wrong format.

[GewoonJaap]

Hmm. But I supplied to same .Wav file to the 64bit library and ARM64 library. It worked fine on 64bit, but displayed commas on the ARM64 library. So I think I must have messed something up in my fork?

[adamnova]

I do not know what changes have you made, but there might be some platform specific issues on ARM, such as your CPU using different endianness or something like that. Unfortunately there is probably infinite number of causes :( You can verify this by just comparing if the loaded values are the same on x86_64 and ARM64.

[GewoonJaap]

https://github.com/GewoonJaap/whisper.cpp/tree/origin/GewoonJaap

This are my changes. The only thing I changed is to make sure it doesn't use the x64 headers.

[adamnova]

I am afraid that there is no easy way to tell whats wrong, you can only try and debug it or wait for official support. It might be that the libraries or something just does not behave correctly on ARM64. I have no idea what exactly could be the cause in this case.

[sandrohanea]

Hello,
Version 1.2.1 was released, and it should contain support for ARM. Can you maybe check it out? https://www.nuget.org/packages/Whisper.net/1.2.1 (I don't have 🪟 on ARM to test it)

[aki-0929]

I has same problem, when i use previous version(1.2.1-preview4) it work fine.
Test server: Linux server 5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13) x86_64 GNU/Linux
StackTrace:
Unhandled exception. System.Exception: Failed to load native whisper library.
at Whisper.net.WhisperFactory..ctor(IWhisperProcessorModelLoader loader, Boolean delayInit)
at Whisper.net.WhisperFactory.FromPath(String path, Boolean delayInitialization)
at SRS.Application.MediaApiController.<>c__DisplayClass6_0.b__0()
at System.Threading.Thread.StartHelper.Callback(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
at System.Threading.Thread.StartCallback()
Aborted

[aki-0929]

I has same problem, when i use previous version(1.2.1-preview4) it work fine. Test server: Linux server 5.10.0-20-amd64 #1 SMP Debian 5.10.158-2 (2022-12-13) x86_64 GNU/Linux StackTrace: Unhandled exception. System.Exception: Failed to load native whisper library. at Whisper.net.WhisperFactory..ctor(IWhisperProcessorModelLoader loader, Boolean delayInit) at Whisper.net.WhisperFactory.FromPath(String path, Boolean delayInitialization) at SRS.Application.MediaApiController.<>c__DisplayClass6_0.b__0() at System.Threading.Thread.StartHelper.Callback(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state) --- End of stack trace from previous location --- at System.Threading.Thread.StartCallback() Aborted

I found the reason, I was missing GLIBCXX_3.4.29.

[sandrohanea]

Indeed, a native library might not be loaded if some dependency is missing (e.g. VS Redistributable on Windows and it seems libc, libm, and libstdc++ on linux.

For macos it is libsystem and libc++.

If the latest version 1.2.1 throws "Failed to load native whisper library" I encourage you to do the following:

On Windows: -Check if you have VS Redistributable 140
-Check with dumpbin /dependents whisper.dll

On Linux: - navigate to runtimes and to specific architecture and run ldd whisper.so => this should display all the dependents and their path (if they are not found, install them)

On MacOS - navigate to runtimes and to specific architecture and run otool -L ./whisper.dylib => same as ldd on linux

[r618]

@sandrohanea I'm getting the error with newest redistributable installed (VS 2022)
VS Redistributable 140 is for VS 2015-2019, and it's not possible to install previous version of the redistributable (installer will refuse to install 140)
[note: I can't remove latest redistributable]
Could be please check if the library is possibly built against specific runtime version, and if so build it against latest available (on user machine) ?
I'm not sure if the above is applicable, at all, I'm just guessing, but thought I'd be worth an ask. Thanks !

[sandrohanea]

Hello @r618, can you maybe try to install: https://aka.ms/vs/17/release/vc_redist.x64.exe?
it is Microsoft Visual C++ Redistributable 2015-2022:

Indeed, the library is built with VS2022
and this is the dumpbin result of the whisper.dll:

[r618]

Hello @r618, can you maybe try to install: https://aka.ms/vs/17/release/vc_redist.x64.exe? it is Microsoft Visual C++ Redistributable 2015-2022:

Indeed, the library is built with VS2022 and this is the dumpbin result of the whisper.dll:

That redistributable is already installed ( i.e. when I run the installer from the link it allows only to modify/repair/uninstall existing install)
The dumpbin is identical (binaries are from latest nuget package)

Settings solution platform explicitly to e.g. 'x64' helped. (all new solutions are by default 'Any CPU'), it loads now, Xeers

(I have another issue with processing a wav file where processor throws an AccessViolationException, i will open another issue possibly)

[GewoonJaap]

Seems to work on Windows for ARM! Thanks!

[MikePittAge]

I've got this exact same problem with win-x64. It works fine on macOS arm64 and Windows arm64, but when I run my code on 64-bit Windows Server 2019 I get "Unhandled exception. System.Exception: Failed to load native whisper library".

I have the latest VC++ runtime installed. It's definitely trying to load whisper.dll from runtimes/win-x64 as if I rename it, it then says it can't find the native dll. It therefore definitely feels like it's a dependency I'm missing.

Grateful for any ideas :)

[sandrohanea]

Hello @MikePittAge ,

Can you, please, double-check that VC++ Redistributable is correctly installed, here is a link to the latest: https://aka.ms/vs/17/release/vc_redist.x64.exe

[MikePittAge]

Hello @MikePittAge ,

Can you, please, double-check that VC++ Redistributable is correctly installed, here is a link to the latest: https://aka.ms/vs/17/release/vc_redist.x64.exe

Thanks @sandrohanea - VC++ Redistributable x86 and x64 are installed. I'll try removing and reinstalling them. I'll also see if I can find a standard Windows client to test on - see if it's related to Server.

[MikePittAge]

Hello @MikePittAge ,

Can you, please, double-check that VC++ Redistributable is correctly installed, here is a link to the latest: https://aka.ms/vs/17/release/vc_redist.x64.exe

I've re-installed VC++ Redistributable and the error is the same. I've tried on 64-bit Windows 11 and it works fine, but I need it to run on Windows Server 2019. Is there a way I can get the underlying exception?

[MikePittAge]

Hi @sandrohanea

I think I have found my issue - in older versions of Window Server (and I assume Windows client) LoadLibrary can fail if the DLL being loaded doesn't have a DLLMain function. It appears that is sorted in Windows Server 2022, Windows 10 & 11 but unfortunately I need to use Windows Server 2019 for now. This Stack article showed me the way -

https://stackoverflow.com/questions/26583503/dll-fails-to-load-in-windows-server-2012

I wrote a quick .net 6 app using LoadLibraryEx using the options the above article suggests and I can load whisper.dll fine. If I just use LoadLibrary, I get a DLL initialisation error.

Is it possible you could update the package to support Windows Server 2019 (and older I guess) in this way?

Thanks,
Mike.

[sandrohanea]

Hey all,
On last commit (e8c49a8) I added support for LoadLibraryEx (which should fix the Windows Server 2019 issue), thanks @MikePittAge for the investigation.

Also added a detailed error message based on the platform so we track different root causes of the loading issue.

Unfortunately, I don't have a Windows Server 2019, so if someone can check it, it will be great.

Tested LoadLibraryEx on Windows 11 and it's working as expected (but LoadLibrary was also working).

[MikePittAge]

Unfortunately, I don't have a Windows Server 2019, so if someone can check it, it will be great.

Thanks @sandrohanea, I'll give it a try now.

[MikePittAge]

@sandrohanea initially it failed but I got the error message this time that a DLL initialisation error occurred. Looking at the code dwFlagsis sent LOAD_LIBRARY_SEARCH_DEFAULT_DIRS, but the stack article suggested DONT_RESOLVE_DLL_REFERENCES. When I send the latter, it loads the DLL fine.

[MikePittAge]

Excellent - thanks so much @sandrohanea.

[sandrohanea]

Version 1.2.2 was released. Which includes all these native libraries, additional information in case the library cannot be loaded, and also the ability to use a custom LibraryLoader.

[lucianomotti]

Hi, just tried the 1.2.2 on AWS EC2 Debian on ARM64 and it still fails:

Unhandled exception. System.Exception: Failed to load native whisper library. Error: Unknown error

Any idea on how to make it work? On the same machine I am able to run whisper.cpp without issues.

Version 1.2.2 was released. Which includes all these native libraries, additional information in case the library cannot be loaded, and also the ability to use a custom LibraryLoader.

[PringlesKing]

I found the reason, I was missing GLIBCXX_3.4.29.

Hey, @aki-0929 how did you install this library? I have the same error, but I can't install it in any way

[aki-0929]

I found the reason, I was missing GLIBCXX_3.4.29.

Hey, @aki-0929 how did you install this library? I have the same error, but I can't install it in any way

I recompiled whistler.cpp and replaced the original file.

[brianhama]

internal class MyCustomWindowsLibraryLoader : ILibraryLoader
{
public LoadResult OpenLibrary(string filename)
{
var loadedLib = LoadLibraryEx(filename, 0x00000001);

    if (loadedLib == IntPtr.Zero)
    {
        var errorCode = Marshal.GetLastWin32Error();
        var errorMessage = new Win32Exception(errorCode).Message;
        return LoadResult.Failure(errorMessage);
    }

    return LoadResult.Success;
}

[DllImport("kernel32", SetLastError = true, CharSet = CharSet.Auto)]
private static extern IntPtr LoadLibraryEx([MarshalAs(UnmanagedType.LPTStr)] string lpFileName, IntPtr hFile, uint dwFlags);

}

Aren't you missing a third parameter when calling LoadLibraryEx in your code?