Natural Subtitle Segmentation and Splitting without trashing the readability.

[ankitgurua]

Natural Subtitle Segmentation and Splitting without trashing the readability.

[ankitgurua]

It works and I understand what you mean. Regarding the lack of timestamp information for numbers, I've been inferring and filling in this data using the timestamps of adjacent words. However, during processing, the original spacy file likely uses the token information from the JSON file generated by Whisper - a feature that WhisperX lacks. I'm uncertain whether this difference impacts Spacy's functionality.

I think i finally found a perfect way to get subtitles, in which i get near perfect aligning of timestamps and the length of the sentence is also naturally segregated. Using this spacy script.

What i did is basically divided what whisperX does in 3 parts.

Transcription, Aligning, Segmenting

What i did is do them all seperately and then combine those different methods to work with each other.

It's a bit long, but its perfect.

[ViiTetrix]

Did you solve this problem?

[ankitgurua]

Did you solve this problem?

Yes i did

Basically some changes in the script made it run with whisperX as well

[ViiTetrix]

So can you teach me how to do this, I try to change it many times,but I failed

[ViiTetrix]

I have only implemented it on whisper. The JSON file output by whisperx does not contain the relevant token information, and I do not know how to handle this part in the script.

[ankitgurua]

I have only implemented it on whisper. The JSON file output by whisperx does not contain the relevant token information, and I do not know how to handle this part in the script.

Yes, i created different spacy files for whisperX and whispertimestamped,
https://gist.github.com/ankitgurua/eac069ed0c95e1ce5924a10923883133
https://gist.github.com/ankitgurua/7b0db06baa8e2c7288cbbf396169120d

the problem with whisperX is that its alignment model cannot align numbers so its json have numbers in sentences but dont have timestamps for it, so to deal with whisperX output you should use --suppress_numerals in your command and then use this spacy script i provided to segment it.

[ViiTetrix]

It works and I understand what you mean. Regarding the lack of timestamp information for numbers, I've been inferring and filling in this data using the timestamps of adjacent words. However, during processing, the original spacy file likely uses the token information from the JSON file generated by Whisper - a feature that WhisperX lacks. I'm uncertain whether this difference impacts Spacy's functionality.

[hongyuhei7722]

What i did is basically divided what whisperX does in 3 parts. Transcription, Aligning, Segmenting
——Could you tell me how to do? Thank you very much.

[ViiTetrix]

Transcription → WhisperX
Aligning → WhisperX -wav2vec2
Segmenting → Spacy

• You can find the script and environmental requirements above