Most of the usage is in the CPU and not in the GPU

[sandrohanea]

Most of the usage is in the CPU and not in the GPU

[sandrohanea]

Hello @yakovw and thanks for the feedback,

Indeed, I observed that CPU is intensively used still, but this is true for whisper.cpp as well:
https://github.com/ggerganov/whisper.cpp#nvidia-gpu-support-via-cublas

Whisper.net is just a wrapper over the native whisper.cpp, while https://github.com/Const-me/Whisper is a port of whisper.cpp with Directx support.

For now, only the decoder is offloaded to the GPU, while the melspectogram and decoder are still running on CPU (same as in the whisper.cpp).

If whisper.cpp will be improved to offload more stuff to GPU, newer versions of whisper.net will automatically get all these improvements.

Will close this issue for now, as the concern is with the native whisper.cpp, not with this wrapper.

[yakovw]

I'm good, thank you very much
Indeed, without a doubt, it is an improvement in every way