spiglerg / rnn_text_generation_tensorflow Goto Github PK
View Code? Open in Web Editor NEWDEPRECATED CODE : Text generation using RNN (LSTM) implemented using Tensorflow
License: Apache License 2.0
DEPRECATED CODE : Text generation using RNN (LSTM) implemented using Tensorflow
License: Apache License 2.0
Hi spiglerg!
First off, I want to say that your code has helped me become much more familiar with the RNN framework inside of Tensorflow. Thank you!
I noticed that on line 47 of rnn_tf.py you create a two-element list of LSTM cells by saying:
[self.lstm_cell]*self.num_layers
Which creates a list of two of the exact same TF objects. Since each TF object has its own set of trainable parameters, won't duplicating the same TF object cause the network to have two layers of LSTM cells that have the exact same parameters?
Hi I am trying to install and I get
Traceback (most recent call last):
File "rnn_tf.py", line 5, in
import argparse
File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 86, in
import copy as _copy
File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/copy.py", line 52, in
import weakref
File "/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/python2.7/weakref.py", line 14, in
from _weakref import (
ImportError: cannot import name _remove_dead_weakref
As you can see after i've terminated the program it doesn't output anything but multiple errors. Same thing happens when i run
$ python rnn_tf.py saved/model.ckpt "The "
although i doesn't happen when i run this code:
$ python rnn_tf.py
another thing that doesn't work is the relative position of the file. So i always have to give the absolute position of data/shakespeare.txt. All of the program runs in terminal Mac version 10.12. İ use python 3.6.2.
İf you need any other information please just ask.
Thank you in advance!
when i want to generate text i get a memory error
Traceback (most recent call last): File "rnn_tf.py", line 300, in <module> main() File "rnn_tf.py", line 221, in main data, vocab = load_data(args.input_file) File "rnn_tf.py", line 174, in load_data data = embed_to_vocab(data_, vocab) File "rnn_tf.py", line 152, in embed_to_vocab data = np.zeros((len(data_), len(vocab))) MemoryError
the text file is ~6000 KB in size but that shouldn't be a problem because i can train with this text.
i am running python in 64-bit
please help!
Hi. Can I use it to generate text including russian words/language? Will it work correctly?
And can u explain me a lil' bit more about how to start generate text?
Cuz' when i choose new textfile to train and started with a command 'python rnn_tf.py' - I have been taking some OpKernels and warnigs + after that waiting for a long time (something like 2-3 hours). There was 800 batches and they didn't want to stop. I stoppped it by myself, cuz' losses was higher, then previos 700 batches.
After that i started with a command ' python rnn_ft.py saved/model.ckpt "The " ' and programm returned me some errors.
Did i did everything right? Mb i should waight till train will be complete? Is it normal, that losses was growing up after 700 batches?
Sorry for my stupid questions and thanks :)
I know my last issue was my fault, but I don't think this one is. I got an argument error about a minute after starting the program. Here's the full error:
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1350, in _do_call
return fn(*args)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1329, in _run_fn
status, run_metadata)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,67] rhs shape= [256,39]
[[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@char_rnn_network/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rnn_network/Variable, save/RestoreV2)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "rnn_tf.py", line 309, in <module>
main()
File "rnn_tf.py", line 261, in main
check_restore_parameters(sess, saver)
File "rnn_tf.py", line 191, in check_restore_parameters
saver.restore(sess, ckpt.model_checkpoint_path)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1686, in restore
{self.saver_def.filename_tensor_name: save_path})
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 895, in run
run_metadata_ptr)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1128, in _run
feed_dict_tensor, options, run_metadata)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1344, in _do_run
options, run_metadata)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1363, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,67] rhs shape= [256,39]
[[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@char_rnn_network/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rnn_network/Variable, save/RestoreV2)]]
Caused by op 'save/Assign', defined at:
File "rnn_tf.py", line 309, in <module>
main()
File "rnn_tf.py", line 257, in main
saver = tf.train.Saver(tf.global_variables())
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1239, in __init__
self.build()
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1248, in build
self._build(self._filename, build_save=True, build_restore=True)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 1284, in _build
build_save=build_save, build_restore=build_restore)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 765, in _build_internal
restore_sequentially, reshape)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 440, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\training\saver.py", line 160, in restore
self.op.get_shape().is_fully_defined())
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\ops\state_ops.py", line 276, in assign
validate_shape=validate_shape)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 62, in assign
use_locking=use_locking, name=name)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3160, in create_op
op_def=op_def)
File "C:\Program Files\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1625, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [256,67] rhs shape= [256,39]
[[Node: save/Assign = Assign[T=DT_FLOAT, _class=["loc:@char_rnn_network/Variable"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](char_rnn_network/Variable, save/RestoreV2)]]
I wanted to train with a fairly large training data file (~60MB). However, I get MemoryErrors. I found out your call to np.zeros() was trying to create an array of 197,159,216,917 elements. Seeing that I don't have 200GB of RAM/Swap, I could see why this would cause a problem. Is there any way to reduce this massive use of memory?
I notice that the longer(the more batches)that I've trained the more perfect that generated sentences will be,but I find that some generated sentences can be totally the same as some sentences in my train corpus,is it possible?I just wonder whether it just generate the sentences like that or 'copy' like that.
Hi,
First, thanks for your work :)
Second, I tried to use your code without any changes for test purposes,, but result isn't good.
Output of python rnn_tf.py saved\model.ckpt "The "
It's a normal result for default configuration or not?
Btw, one time I tried to clear saved
folder, and after train file model.ckpt
didn't create, only checkpoint/index/meta/data files. It's right behavior or I missed something?
My env:
Windows 10 x64
Python 3.5.2
Tensorflow GPU 1.1.0 (works with GeForce GTX 950M)
I didn't see so quickly any other way to contact you, but I wanted to give credit. I re-used your model with some slight adjustments to generate icons and hieroglyphs instead of text:
http://blog.douweosinga.com/2017/01/learning-to-draw-generating-icons-and.html
Hi:
Your script is generating very good training errors below 0.4. after 60k batches Is there a snippet of code that can be added to include a temperature option common to other lstm packages to improve output?
Thanks
I tried to generate German text. Unfortunately the results are only extracts from the original text. But no generated new remixes.
For testing i used Wagner's Lohengrin (old German) and German Hip Hop lyrics (of one group). In order to verify i successfully tried Vernor Vinges Rainbows End in English language. #
First thanks for this work. Works very well.
Whilst I'm testing it, it issues a warning:
WARNING:tensorflow:<tensorflow.python.ops.rnn_cell_impl.BasicLSTMCell object at 0x119c13748>: Using a concatenated state is slower and will soon be deprecated. Use state_is_tuple=True.
tensorflow version 1.3.0
Python 3.6.2
Anyway to 'fix' it?
Nice work! It is very clear and easy to understand!
I am wondering how could I change it to a word-level Generator?
Hi. It's me again :)
K, i used command $ python rnn_tf.py and some hours ago it was finished. And what should i do now?
Should i use command $ python rnn_tf.py saved/model.ckpt "The " to generate my text?
Will it generate a new file at "saved" directory ? How can i find text file?
Or mb i need to use another prefix? I didn't understand it yet :( Can u explain me a little, please?
Thx a lot and sorry for giving u some work :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.