danieldjohnson/biaxial-rnn-music-composition

A recurrent neural network designed to generate classical music.

biaxial-rnn-music-composition's Issues

Question

I dont know anything about code writing..and this seems pretty complicated for me ...do i have to add code in the model? like create an output folder? idk how to do this ...i am sorry to bother but can no one put this in an easy out of the box executable code? or make a readme for dummies (like me)?
I tried 2 AI composers ..but one only generates same note length and the other one don´t lets me input own midis...while the results of this one seem outstanding! i really wanna have it =,O [sry to bother]

Any way to train on CPU?

Is it possible to train the network on CPU instead of GPU? And how long would it possibly take?

IOError: [Errno 2] No such file or directory: 'output/sample0.mid'

Can not use cuDNN on context None: cannot compile with cuDNN. We got this error:
/usr/bin/ld: cannot find -lcudnn
collect2: error: ld returned 1 exit status

Mapped name None to device cuda: GeForce GTX 750 Ti (0000:01:00.0)
Loaded DEB_PASS
model.py:370: UserWarning: Updating an OrderedUpdates with a non-ordered dictionary with 2+ elements could make your code non-deterministic
self.walk_input: next_input
model.py:373: UserWarning: Updating an OrderedUpdates with a non-ordered dictionary with 2+ elements could make your code non-deterministic
updates.update({hidden:newstate for hidden, newstate, layer in zip(self.walk_hiddens, new_states, self.time_model.layers) if has_hidden(layer)})
epoch 0, error=15666.3876953

IOError Traceback (most recent call last)
in ()
44 m = model.Model([300,300],[100,50], dropout=0.5)
45
---> 46 multi_training.trainPiece(m, pcs, 10000)
47
48 pickle.dump( m.learned_config, open( "output/final_learned_config.p", "wb" ) )

/media/yzy1001/pylearning/jupyter_notebook/biaxial-rnn-music-composition-master/multi_training.pyc in trainPiece(model, pieces, epochs, start)
56 if i % 500 == 0 or (i % 100 == 0 and i < 1000):
57 xIpt, xOpt = map(numpy.array, getPieceSegment(pieces))
---> 58 noteStateMatrixToMidi(numpy.concatenate((numpy.expand_dims(xOpt[0], 0), model.predict_fun(batch_len, 1, xIpt[0])), axis=0),'output/sample{}'.format(i))
59 pickle.dump(model.learned_config,open('output/params{}.p'.format(i), 'wb'))
60 signal.signal(signal.SIGINT, old_handler)

/media/yzy1001/pylearning/jupyter_notebook/biaxial-rnn-music-composition-master/midi_to_statematrix.pyc in noteStateMatrixToMidi(statematrix, name)
99 track.append(eot)
100
--> 101 midi.write_midifile("{}.mid".format(name), pattern)

/home/yzy1001/anaconda3/envs/tensorflow_py27/lib/python2.7/site-packages/midi/fileio.pyc in write_midifile(midifile, pattern)
148 def write_midifile(midifile, pattern):
149 if type(midifile) in (str, unicode):
--> 150 midifile = open(midifile, 'wb')
151 writer = FileWriter()
152 return writer.write(midifile, pattern)

IOError: [Errno 2] No such file or directory: 'output/sample0.mid'

Could someone tell me why is it requiring a output file or something, thank you.

Add support for eXtended Module format

While training from MIDI files is good, there's a vast tracker music library, with absolute majority being xm format files. It might be interesting to experiment with creation of tracker-style music with neural network.

There's a xm to midi converter on the net, but it isn't any good. However, it's possible to read xm files directly and there's even ready solution for that in python:

https://sourceforge.net/p/xmp/libxmp-python/ci/master/tree/examples/

Not only it allows to read xm files with python, but it also support several dozens of tracker music formats. Entire modarchive is about 50 GB of music and most of it will be readable with libxmp-python.

ValueError: When compiling the inner function of scan the following error has been encountered

this error occurs at
git remote -v
origin https://github.com/bazarevsky/biaxial-rnn-music-composition (fetch)
origin https://github.com/bazarevsky/biaxial-rnn-music-composition (push)

rzai@rzai00:~/prj/biaxial-rnn-music-composition$ CUDA_VISIBLE_DEVICES=1 python main.py 2>&1 | tee yknote---main.py---log
Using gpu device 0: GeForce GTX 1080 (CNMeM is disabled, cuDNN 5005)
Loaded chpn-p14_format0
Loaded bk_xmas2_format0
Loaded grieg_waechter
Loaded scn15_7_format0

...................

Loaded mendel_op30_5_format0
Loaded chpn_op35_1
Loaded deb_prel
Traceback (most recent call last):
File "main.py", line 52, in
m = model.Model([300,300],[100,50], dropout=0.5)
File "/home/rzai/prj/biaxial-rnn-music-composition/model.py", line 101, in init
self.setup_predict()
File "/home/rzai/prj/biaxial-rnn-music-composition/model.py", line 315, in setup_predict
n_steps=self.steps_to_simulate )
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan.py", line 761, in scan
condition, outputs, updates = scan_utils.get_updates_and_outputs(fn(*args))
File "/home/rzai/prj/biaxial-rnn-music-composition/model.py", line 295, in step_time
notes_result, updates = theano.scan(fn=self._predict_step_note, sequences=[time_final], outputs_info=note_outputs_info)
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan.py", line 1058, in scan
scan_outs = local_op(*scan_inputs)
File "/usr/local/lib/python2.7/dist-packages/theano/gof/op.py", line 604, in call
node = self.make_node(*inputs, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/theano/scan_module/scan_op.py", line 539, in make_node
inner_sitsot_out.type.dtype))
ValueError: When compiling the inner function of scan the following error has been encountered: The initial state (outputs_info in scan nomenclature) of variable IncSubtensor{Set;:int64:}.0 (argument number 3) has dtype int16, while the result of the inner function (fn) has dtype int8. This can happen if the inner function of scan results in an upcast or downcast.
rzai@rzai00:~/prj/biaxial-rnn-music-composition$

Python AssertionError

When I run "m = model.Model([300,300],[100,50], dropout=0.5)" I get a lot of this error :

<<!! BUG IN FGRAPH.REPLACE OR A LISTENER !!>> <type 'exceptions.Exception'> Constant{300} is already owned by another fgraph local_elemwise_alloc
ERROR (theano.gof.opt): Optimization failure due to: local_elemwise_alloc
ERROR (theano.gof.opt): node: Elemwise{add,no_inplace}(dot.0, Alloc.0)
ERROR (theano.gof.opt): TRACEBACK:
ERROR (theano.gof.opt): Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/theano/gof/opt.py", line 1822, in process_node
    fgraph.replace_all_validate(repl_pairs, reason=lopt)
  File "/usr/lib/python2.7/site-packages/theano/gof/toolbox.py", line 306, in replace_all_validate
    fgraph.replace(r, new_r, reason=reason, verbose=False)
  File "/usr/lib/python2.7/site-packages/theano/gof/fg.py", line 592, in replace
    self.change_input(node, i, new_r, reason=reason)
  File "/usr/lib/python2.7/site-packages/theano/gof/fg.py", line 530, in change_input
    self.__import_r__(new_r, reason=reason)
  File "/usr/lib/python2.7/site-packages/theano/gof/fg.py", line 363, in __import_r__
    self.__import__(variable.owner, reason=reason)
  File "/usr/lib/python2.7/site-packages/theano/gof/fg.py", line 394, in __import__
    raise Exception("%s is already owned by another fgraph" % r)
Exception: Constant{300} is already owned by another fgraph

and the process is never finish (waiting about two hours). I have changed the recursion limit with sys.setrecursionlimit() to 10.000 because an exception about recursion limit appears. But nothing.

I have an I7 core, are these errors normal ?

Output tends to converge to being identical to one particular piece

In training small batches of music (~30), I realized that the output of the network especially further in the training process, tend to directly plagiarize one of the input pieces. This issue is particularly apparent when training with small EDM segments, which are essentially repeating chords. Is there a way to modify the network such that this problem is avoided?

Just for reference, I upsized the network by 2x the original sizes in blind attempt to produce a better result. Not sure if that is a root cause to this issue.

Error

Hi I am getting the following error:
ValueError: ('The following error happened while compiling the node', forall_inplace,gpu,scan_fn}(Elemwise{switch,no_inplace}.0, Subtensor{int64:int64:int8}.0, GpuIncSubtensor{InplaceSet;:int64:}.0, GpuIncSubtensor{InplaceSet;:int64:}.0, Elemwise{switch,no_inplace}.0, Layer.linear_matrix, Layer.linear_matrix, Layer.linear_matrix, Layer.linear_matrix, Layer.linear_matrix, Layer.linear_matrix, Layer.linear_matrix, Layer.linear_matrix, GpuFromHost.0, GpuFromHost.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0, GpuDimShuffle{0,x}.0), '\n', 'numpy.dtype has the wrong size, try recompiling')

For the following line
self.update_fun = theano.function(
inputs=[self.input_mat, self.output_mat],
outputs=self.cost,
updates=updates,
allow_input_downcast=True)

Do you have any idea why?

Correctly aligned MIDIs?

Hi,

I have been testing the system with various collections of MIDIs, each from one renowned classical composer (e.g. Bach, Mozart...). All of the MIDIs were obtained from www.midiworld.com.

Unfortunately all of the outputs appear just to be a sequence of random notes, with extremely rapid tempo. I reckon that's due to those inputs' quality. Therefore I am wondering if you can share the MIDIs you used originally for generating those amazing compositions on your website?

Thanks in advance!

model

Hello

When I try to run
m = model.Model([300,300],[100,50], dropout=0.5)

from python shell program doesn't give me any response (even error message)

Could you help me?

Thank you

Question about BEAT input

Hello, I'm Trying to understand precisely how it all works and do my own version from ground up.

However, I don't understand how exactly beat input works. It has fixed pattern. Can I observe that pattern in the note sheet ?
How do we come up with this exact pattern for 4/4 ?

Why the beat has 4 inputs ? ( rows)

It would be very helpful if someone could explain this to me in more details. I do a Bachelor thesis and I'm kind of stuck with this bit.

Any way to resume training?

Is it possible to resume training in any way?

would you guide me to get some dataset(music)?

Weird behaviour

I tried to run the app with various amount of different midi samples, however after files are loaded I am getting this warning
/model.py:372: UserWarning: Updating an 'OrderedUpdates' with a non-ordered dictionary with 2+ elements could make your code non-deterministic self.walk_input: next_input /model.py:375: UserWarning: Updating an 'OrderedUpdates' with a non-ordered dictionary with 2+ elements could make your code non-deterministic updates.update({hidden:newstate for hidden, newstate, layer in zip(self.walk_hiddens, new_states, self.time_model.layers) if has_hidden(layer)})

and after that it fails with Memory error
Traceback (most recent call last): File "main.py", line 46, in <module> multi_training.trainPiece(m, pcs, 10000) File "/LSTM-generator/multi_training.py", line 53, in trainPiece error = model.update_fun(*getPieceBatch(pieces)) File "/home/.conda/envs/music-generation/lib/python2.7/site-packages/theano/compile/function_module.py", line 871, in __call__ storage_map=getattr(self.fn, 'storage_map', None)) File "/home/.conda/envs/music-generation/lib/python2.7/site-packages/theano/gof/link.py", line 314, in raise_with_op reraise(exc_type, exc_value, exc_trace) File "/home/.conda/envs/music-generation/lib/python2.7/site-packages/theano/compile/function_module.py", line 859, in __call__ outputs = self.fn() File "/home/.conda/envs/music-generation/lib/python2.7/site-packages/theano/scan_module/scan_op.py", line 951, in rval r = p(n, [x[0] for x in i], o) File "/home/.conda/envs/music-generation/lib/python2.7/site-packages/theano/scan_module/scan_op.py", line 940, in <lambda> self, node) File "theano/scan_module/scan_perform.pyx", line 524, in theano.scan_module.scan_perform.perform (/home/.theano/compiledir_Linux-2.6-el6.x86_64-x86_64-with-redhat-6.8-Carbon-x86_64-2.7.13-64/scan_perform/mod.cpp:5862) File "/home/.conda/envs/music-generation/lib/python2.7/site-packages/theano/tensor/type.py", line 634, in value_zeros return numpy.zeros(shape, dtype=self.dtype) MemoryError

I tried to decrease the batch_width and run on both cpu and gpu clusters (there was no warning about memory over-usage from clusters though) so I am kinda lost. What could be an issue?

doesn't work, need help

everytime i've tried it we got this :
musicbots@music:~/biaxial-rnn-music-composition$ python main.py
Loaded bwv798
Loaded bwv783
Loaded bwv781
Loaded bwv796
Loaded bwv797
Loaded bwv800
Loaded bwv789
Loaded bwv776
Loaded bwv784
Loaded bwv779
Loaded bwv793
Loaded bwv792
Loaded bwv794
Loaded bwv795
Loaded bwv782
Loaded bwv788
Loaded bwv772
Loaded bwv775
Loaded bwv791
Loaded bwv801
Loaded bwv774
Loaded bwv787
Loaded bwv780
Loaded bwv786
Loaded bwv790
Loaded bwv778
Loaded bwv777
Loaded bwv799
Loaded bwv785
Loaded bwv773
/home/musicbots/biaxial-rnn-music-composition/model.py:370: UserWarning: Updatin g an OrderedUpdates with a non-ordered dictionary with 2+ elements could make your code non-deterministic
self.walk_input: next_input
/home/musicbots/biaxial-rnn-music-composition/model.py:373: UserWarning: Updatin g an OrderedUpdates with a non-ordered dictionary with 2+ elements could make your code non-deterministic
updates.update({hidden:newstate for hidden, newstate, layer in zip(self.walk_h iddens, new_states, self.time_model.layers) if has_hidden(layer)})
Killed

we tried it on multiple machines, as well as digital ocean always comes back to this, can anyone help me getting this to work? i would really like it to see it working

How to load weights?

How may I load already trained weights to the model?

Issue with local_bitwidth()

Following error occurs when running app on 64bit system
model.py", line 21, in matrixify n if (theano.gof.local_bitwidth() == 64) else T.cast(n,'int32'), AttributeError: 'module' object has no attribute 'local_bitwidth'

However, if change code in module.py from
return T.repeat(T.shape_padleft(vector), n if (theano.gof.local_bitwidth() == 64) else T.cast(n,'int32'), axis=0)
back to
return T.repeat(T.shape_padleft(vector), n, axis=0)
no error occurs.

Error

I'm getting ValueError: empty range for randrange()

Any way to set output size?

I have been using this for a while now, but can't seem to work out how to change the output size. Is there some hidden config file, or will I need to edit the code?

Multiple track support

@hexahedria, can you predict what happens if you feed midi files which have multiple tracks? Is it supported, does it select the first track of the midi file?

If there is no support, I'm ready to take up development and add this feature...

Allocation error

I am getting the following error

RuntimeError: CudaNdarray_ZEROS: allocation failed.

is this because of theano version?

Thanks

Failed to alloc

I ran codes without any change, but error info as follow, could you give some suggestion:

numpy.repeat TypeError on 32 bit systems

Traceback (most recent call last):
...
self.setup_train()
time_outputs_info = [initial_state_with_taps(layer, num_time_parallel) for layer in self.time_model.layers]
...
return matrixify(layer.initial_hidden_state, dimensions) if has_hidden(layer) else None
return T.repeat(T.shape_padleft(vector), n, axis=0)
...
TypeError: ("dtypes ('uint32', 'int64', 'uint64') are not supported by numpy.repeat for the 'repeats' parameter, ", 'int64')

dtype int16 instead of dtype int8 in outputs_info

ValueError: When compiling the inner function of scan the following error has been encountered: The initial state (outputs_info in scan nomenclature) of variable IncSubtensor{Set;:int64:}.0 (argument number 3) has dtype int16, while the result of the inner function (fn) has dtype int8. This can happen if the inner function of scan results in an upcast or downcast.

OSX 10.11.6
CUDA Driver Version: 8.0.46
python 2.7

BUG with this program

issue with 'out of memory'

I have run your code with GPU but met a problem with "out of memory" like in http://deeplearning.net/software/theano/faq.html "Out of memory... but not really". Is it ok for you to use a gpu to run this code ?

Longer sample mids

Hello, I've been testing this and it works well, but I would like to have longer sample midis. It currently creates midis of only 9-10 measures. Does the code need to be edited? where? I can't seem to figure it up.

Thank you

A question about Bi-axial LSTM structure

Why choose to set time-axis ahead and note-axis behind rather than set note-axis ahead and time-axis behind.