char lm using the default config

[rwth-i6]

char lm using the default config

[albertz]

Are you using LmDataset? Check the documentation and also the code to understand all the options. For debugging / understanding how you configured the dataset, you can use tools/dump-dataset.py, which will just dump parts of the dataset. Then you see if the data is as you intended it to be.

[kazuki-irie]

How to add space in the vocab

With the current version of the code, you can only do that by pre-processing the data (split words to space separated characters), explicitly add end-of-word <eow> tokens, and use "word_based": True.

Also, after first epoch it shows dev_score_output:exp': 2.9312321849827905 which doesn't seem reasonable.

Why do you think so? I do not know how large your epoch is in terms of number of updates but given the character vocabulary size of 39, it looks rather reasonable.

[deep-speech]

Thanks for the reply.
I am training it on LibriSpeech LM corpus with epoch split of 4, "word_based": False. I replaced space in the corpus with '_' and added it to the vocab.

[kazuki-irie]

I see. That should also work as '_' does not appear in the original LibriSpeech texts.
Closing.