dhimasryan/mosa-net-cross-domain

Hi, I am Ryandhimas Zezario

I am currently doing research in the area of speech processing, focusing on deep learning-based speech assessment models, and speech enhancement.

I am looking forward to collaborating in the area of speech processing.

📫 How to reach me

mosa-net-cross-domain's Issues

Config problem

Hi, I got ConfigAttributeError problem when i running this baseline for MOS challenge.
image

Do you know where is the problems?
Thank you so much.

Can you provide the structure of the txt files

Can you provide the structure of the .txt files of Test_list_PS, Test_list_End2End, Test_list_hubert.
I am trying to test some data on your pre-trained model but I can't understand what is the information I need to input.

Extracting feature fails on RTX2080ti due to insufficient memory size

The command

 python3 Extracting_Hubert_Feature_VoiceMOS_Challenge.py --track phase1-main

fails due to insufficient memory size on RTX2080ti, but the program works when forward propagation is wrapped around with torch.no_grad().

I'll send PR regarding this issue

fairseq should be included as submodule

I have confirmed that the program doesn't run with the latest fairseq.
To avoid this confusion the fairseq repository should be included as submodule

error in pre trained model

I am using the MOSA-Net_Cross_Domain_epoch_100.h5 pre-trained model to test my data but it is showing the following error:

ValueError: Layer count mismatch when loading weights from file. Model expected 21 layers, found 17 saved layers.

Generating SSL features (loading Hubert model)

Hiya, thanks for the nice paper & code 👏 👏

I'm trying to test out the pretrained model, for which I'm generating SSL features using this code

I have fairseq==0.10.1 installed which matches the fairseq-lib submodule SHA in the repo and I can import it into py36 no-problem. The problem comes 👇 👇 when I try to load the HUBERT model, which I downloaded from https://github.com/facebookresearch/fairseq/tree/main/examples/hubert. I'm not sure if the arg_overrides are compatible, as the pretrained model dictionary does not a data key and I get AttributeError: 'NoneType' object has no attribute 'data', when running the load_model_ensemble_and_task

def Save_NPY(Train_data, Val_data, track_name):
    cp_path = './fairseq/hubert_large_ll60k.pt'
    model, cfg, task = fairseq.checkpoint_utils.load_model_ensemble_and_task([cp_path], 
                                    arg_overrides={"data": "./fairseq/data/dict"})

I tried removing arg_overrides to learn more & explore the pretrained models dictionary but found it full of issues which led me to all the various issues people have seen with loading the pretrained model (1, 2)

Do you have any guidance, updated example-code (or versions) or even the exact version of the pretrained model (or dictionary) that can be used to generate/reproduce the SSL features for MOSA-Net? Thank you very much 🙏