What is EvaluationTokenizer?

[EmbraceAir]

What is EvaluationTokenizer?

[faychu]

Sorry, it is now uploaded here:
https://github.com/QwenLM/Qwen-Audio/blob/main/eval_audio/evaluate_tokenizer.py

[EmbraceAir]

@faychu Thanks for ur update. The https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Audio/evaluation/covost2_eval.jsonl is also unavailable.

Besides, what is the correct prompt for speech2text translation? The <|startoftranscription|><|en|><|translate|><|de|><|notimestamps|><|itn|> doesn't work during my evaluation, which only generates the English trans with time stamps.

[faychu]

We have re-uploaded the file https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen-Audio/evaluation/covost2_eval.jsonl, please take a look again.

Yes, The <|startoftranscription|><|en|><|translate|><|de|><|notimestamps|><|itn|> is the correct prompt. However, in a zero-shot setting, the pretrain model Qwen-Audio sometimes struggles to consistently follow the text instructions.