The whisper-large-v3 model randomly misses sentences during recognition when return_timestamps="word"

[zxl777]

The whisper-large-v3 model randomly misses sentences during recognition when return_timestamps="word"

[naveen-corpusant]

I've run into this as well- one unblock I found (haven't tracked why this is the case), is that if you also include return_language=True in your pipe (so have both return_language=True, return_timestamps="word"), then the word level timestamps are correct / make sense. We were seeing some pretty nonsense timestamps without this, it could be the case that some other intermediate reps as needed to properly time align, and are only getting passed through when language info is being passed

[zxl777]

Thank you for your response.
Even after I added return_language=True, the issue still persists.
This parameter does not affect the problem I've encountered.

I've run into this as well- one unblock I found (haven't tracked why this is the case), is that if you also include return_language=True in your pipe (so have both return_language=True, return_timestamps="word"), then the word level timestamps are correct / make sense. We were seeing some pretty nonsense timestamps without this, it could be the case that some other intermediate reps as needed to properly time align, and are only getting passed through when language info is being passed

[amyeroberts]

also cc @ylacombe

[zxl777]

Any update?