juliuskunze/speechless

Speech-to-text based on wav2letter built for transfer learning

speechless's Issues

Errors in tensorflow version 2.1.0

Hallo I have some problems to use your program.

I could solve some of the errors:

In net.py you have to change:

tf.log to tf.math.log

tensorflow.to_int32 to tf.cast(x, tf.int_32)

tf.sparse_to_dense to tf.sparse.to_dense(st, default_value=-1)

But i am not able to solve this error:

return ops.EagerTensor(value, ctx.device_name, dtype)
TypeError: Cannot convert 0.0 to EagerTensor of dtype int32

which happens here:

    return ops.EagerTensor(value, ctx.device_name, dtype)
TypeError: Cannot convert 0.0 to EagerTensor of dtype int32

Thanks

Output only blank

Hi, I tried to training from scratch using minimal english. The output quickly converged to underscore, so all the predicted transcripts are just empty strings. Any suggestions? Thank you.

Pre-trained model

Thank You for providing the codes and pretrained models..

The output of the provided English pretrained model is not very good..can you please provide the model pretrained on 1000h LibriSpeech corpus??

Thanks Again..

Correct way to do transfer learning (Keras AssertionError)

Hi, I'm trying to do transfer learning from the provided English net to a custom data set. The readme mentions that even languages with a different number of characters compared to English can be used by supplying the allowed_characters_for_loaded_model parameter, however I'm getting an AssertionError upon execution:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    english = Configuration.english().load_model(load_name=Configuration.english_baseline[0], load_epoch=Configuration.english_baseline[1], allowed_characters_for_loaded_model=new_corpus.frequent_characters)
  File "/usr/local/lib/python3.5/dist-packages/speechless/configuration.py", line 184, in load_model
    reinitialize_trainable_loaded_layers=reinitialize_trainable_loaded_layers)
  File "/usr/local/lib/python3.5/dist-packages/speechless/net.py", line 182, in __init__
    loaded_first_layers_count=frozen_layer_count if reinitialize_trainable_loaded_layers else None)
  File "/usr/local/lib/python3.5/dist-packages/speechless/net.py", line 231, in load_weights
    asg_transition_probabilities=self.asg_transition_probabilities)
  File "/usr/local/lib/python3.5/dist-packages/speechless/net.py", line 182, in __init__
    loaded_first_layers_count=frozen_layer_count if reinitialize_trainable_loaded_layers else None)
  File "/usr/local/lib/python3.5/dist-packages/speechless/net.py", line 212, in load_weights
    self.predictive_net.load_weights(str(load_model_from_directory / self.model_file_name(load_epoch)))
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/network.py", line 1161, in load_weights
    f, self.layers, reshape=reshape)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/saving.py", line 915, in load_weights_from_hdf5_group
    reshape=reshape)
  File "/usr/local/lib/python3.5/dist-packages/keras/engine/saving.py", line 571, in preprocess_weights_for_loading
    assert shape[0] == layer.filters and shape[2:] == (layer.kernel_size[0], 1)
AssertionError

I'm starting the training by this script:

english = Configuration.english().load_model(load_name=Configuration.english_baseline[0], 
               load_epoch=Configuration.english_baseline[1], 
               allowed_characters_for_loaded_model=new_corpus.frequent_characters)
new = Configuration.new()
new.train(english, run_name="transfer_test_01")

Training from scratch has no problems. What am I doing wrong?

No .wav file is generated

I have downloaded english model and placed it in appropriate directory , I can see that model loads successfully , and audio input is working , but it never ends , it keeps detecting background sounds and still doesnt generate any .wav file , I have to manually press ctrl ^ c to stop it after few minutes .

Test on single .wav file

Hi, It works really well for the recorded audios. Could you tell how can I test it on a given .wav file ?

Does not exist?

Hello. I recently tried installing your file via the pip3 instructions on your github.

pip3 install [email protected]:JuliusKunze/speechless.git

Unfortunately, I seem to be getting this error:

Invalid requirement: '[email protected]:JuliusKunze/speechless.git'
It looks like a path. File '[email protected]:JuliusKunze/speechless.git' does not exist.

Did you remove your github from the system. If so, is there another way to install this?

German Corpus

I was trying to find German corpus but couldn't find it, can you share that please!!!! It would be great help

Updates on ASG loss?

Hello,
Great work on transfer learning. Wanted to know if you guys are planning to share models trained on ASG loss. I saw some work on ASG loss function inside the scripts. Is is supported currently?
Thanks.