ml5js/training-charrnn

Training charRNN model for ml5js

training-charrnn's Issues

UnicodeDecodeError

Getting the following message while training:

File "train.py", line 179, in
main()
File "train.py", line 76, in main
train(args)
File "train.py", line 91, in train
data_loader = TextLoader(args.data_dir, args.batch_size, args.seq_length)
File "/spell/training-lstm/utils.py", line 21, in init
self.preprocess(input_file, vocab_file, tensor_file)
File "/spell/training-lstm/utils.py", line 30, in preprocess
data = f.read()
File "/usr/lib/python3.5/codecs.py", line 698, in read
return self.reader.read(size)
File "/usr/lib/python3.5/codecs.py", line 501, in read
newchars, decodedbytes = self.decode(data, self.errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x92 in position 7047: invalid start byte

TensorFlow 2.0 compatibility?

I believe this training script only works with tensorflow 1.x so just noting that we should test / upgrade for compatibility with tensorflow 2.0 at some point.

Add training resources / tutorials

Paperspace:
https://blog.paperspace.com/training-an-lstm-and-using-the-model-in-ml5-js/

Spell:
https://youtu.be/xfuVcfwtEyw

Interrupting and Resuming Training

Is it possible to interrupt training, and then resume it later?

Best case scenario, I train it on one machine, interrupt, and then continue training on another. No GPU involved, CPU only.

Or is it possible to interrupt training, and save to a model immediately, before it has trained completely? When I interrupt now, no Models folder is created, to save the models.

Thanks

does training-charRNN use GPU or CPU

and if it uses CPU how do I set it to GPU

Training "Cache"?

When I tried to continually train models with different text files, without moving out or deleting generated models, checkpoints etc., the later model files got will always be exactly the same as the first one trained - and generate the same style texts as the first one, even though they are placed in different folders (named after the original text files) in models/.

I put all the original .txt files in one folder in the root of the repo folder. Haven't try if this can be resolved if they're placed in different folders.

How to make it learn to rhyme?

Trining on big files 25+ MB gets killed

I'm training the LSTM with some 80 MB files with the specified hyperparameters

python train.py --data_dir=./data    --rnn_size 2048    --num_layers 2   --seq_length 256 --batch_size 128 --output_keep_prob 0.25

but after few minutes the job gets killed.
Is the file too big?

README clarification on hyperparameters

The suggested hyperparameters have a dropout of 0.25. However, it's unclear which option to use with train.py in order to achieve this?

...
parser.add_argument('--output_keep_prob', type=float, default=1.0,
  help='probability of keeping weights in the hidden layer')
parser.add_argument('--input_keep_prob', type=float, default=1.0,
  help='probability of keeping weights in the input layer')

My guess would be either output_keep_prob or input_keep_prob but I'm not sure.

Thanks!

Uncaught TypeError: ml5.LSTMGenerator is not a function

Seems that README.md needs small update for the latest version of ml5js.

I guess for the latest version, we need to use charRNN instead of LSTMGenerator.
Actually the following works to me.

const lstm = ml5.charRNN('./dataset/', modelLoaded);

tf-core.esm.js:17 Uncaught TypeError: Y is not a function

    tf-core.esm.js:17 Uncaught TypeError: Y is not a function
    at new e (tf-core.esm.js:17)
    at Function.e.make (tf-core.esm.js:17)
    at nt (tf-core.esm.js:17)
    at Object.it (tf-core.esm.js:17)
    at Object.<anonymous> (postprocess.js:11)
    at n (bootstrap:19)
    at Object.<anonymous> (index.js:18)
    at n (bootstrap:19)
    at Object.<anonymous> (index.js:12)
    at n (bootstrap:19)```

Training on word level

First of all: thanks for doing this!
It's an awesome achievement in terms of making these tools accessible to everybody. 🎨🖌

I'm trying to create a model based on a series of poems, is there a way to train it on words?
I would like to compare the results of training the rnn on char sequences or word sequences.

Thanks!

typo in about

it currently says
"Training charRNN modek with ml5js "
there is a "k" in "model" that could be fixed :)