How to convert whisper model to GGML

[yakovw]

How to convert whisper model to GGML

[sandrohanea]

No, there is no way to do it in C# nor a plan to implement it in the future. Do you see any reason why someone would need to do it in C#?

However, Whisper.net have a downloader, which is using huggingface in order to be easy to get a GGML model (either quantized or not): https://github.com/sandrohanea/whisper.net/blob/main/Whisper.net/Ggml/WhisperGgmlDownloader.cs

Of course, one can just create a whisper GGML model using the instructions here: https://github.com/ggerganov/whisper.cpp/blob/master/models/README.md

But even those, are using python, and are not implemented in C++.

[yakovw]

You're right
But I'm trying to convert existing models and can't, and there are new models that are more than there are in the above path
I wanted to convert these models
https://huggingface.co/Shiry/whisper-large-v2-he/tree/main
The script failed
Because he don't know how to convert Transformers models

[sandrohanea]

I tried that fine-tuned model using https://github.com/ggerganov/whisper.cpp/blob/master/models/README.md#fine-tuned-models and it was successfully generated:

Check this command:

git clone https://github.com/openai/whisper
git clone https://github.com/ggerganov/whisper.cpp
git clone git clone https://huggingface.co/Shiry/whisper-large-v2-he

python3 ./whisper.cpp/models/convert-h5-to-ggml.py ./whisper-large-v2-he/ ./whisper .

Please, ensure you have installed and enabled git-lfs in order to clone that huggingface repo.
Also, make sure you have python3 with torchvision and transformers.

[yakovw]

Wait, if you managed to convert it, then it's best for me to take what you managed, is there a way to get to it?

[sandrohanea]

It's a pretty big file (~3gb) so cannot upload it here, maybe I'll add multiple fine-tuned versions to https://huggingface.co/sandrohanea/whisper.net/tree/main but for now, it's probably faster for you to execute the commands above.

If I'll add them to huggingface (+ add some readme on each one with the attribution for each owner), I can modify the downloader in whisper.net to help download these finetunes as well.

[yakovw]

Well I'll try, if I don't succeed I'll wait for you to upload it...
Thanks for bothering to try at all.
Let's hope for the best

[yakovw]

It's a pretty big file (~3gb) so cannot upload it here, maybe I'll add multiple fine-tuned versions to https://huggingface.co/sandrohanea/whisper.net/tree/main but for now, it's probably faster for you to execute the commands above.

If I'll add them to huggingface (+ add some readme on each one with the attribution for each owner), I can modify the downloader in whisper.net to help download these finetunes as well.

I've been trying for a long time, all I manage to produce is the next line
Usage: convert-h5-to-ggml.py dir_model path-to-whisper-repo dir-output [use-f32]
He is in no way willing to convert the model for me

[sandrohanea]

I thought about it and won't put finetunes to the downloader and hugging face: there are too many and we cannot be sure that everything is working as designed). Ofc, if someone wants, they can build them. I think you missed some of the directories in the arguments of convert-h5-to-ggml.py.
However, I uploaded this temporary to my one drive: https://1drv.ms/u/s!AnblcWiFT8S5gas2R15cv_PIZxEk6w?e=qd8Dbg.

Please, react to this message so that I can remove it afterwards. (Otherwise, it will be deleted on 2023-06-25)

[yakovw]

Thank you very much, that is very nice of you.
I downloaded it, you can delete.
I am now checking if it actually works well

I thought about it and won't put finetunes to the downloader and hugging face: there are too many and we cannot be sure that everything is working as designed). Ofc, if someone wants, they can build them. I think you missed some of the directories in the arguments of convert-h5-to-ggml.py. However, I uploaded this temporary to my one drive: https://1drv.ms/u/s!AnblcWiFT8S5gas2R15cv_PIZxEk6w?e=qd8Dbg.

Please, react to this message so that I can remove it afterwards. (Otherwise, it will be deleted on 2023-06-25)