brunnergino/midi-vae

MIDI-VAE

Paper

MIDI-VAE: MODELING DYNAMICS AND INSTRUMENTATION OF MUSIC WITH APPLICATIONS TO STYLE TRANSFER

Paper accepted at 19th International Society for Music Information Retrieval Conference (ISMIR), Paris, France, September 2018

Music Samples

www.youtube.com/channel/UCCkFzSvCae8ySmKCCWM5Mpg

Dataset

All the music pieces we used for generating the audio samples on Youtube and the evaluation in the paper can be downloaded here: https://goo.gl/sNpgQ7

Preparation

• Install common libraries like numpy matplotlib pickle numpy progressbar sklearn scipy csv keras tensorflow theano (some functions are only supported with theano because of recurrentshop)

• Make sure you have installed the following packages https://github.com/craffel/pretty-midi https://github.com/farizrahman4u/recurrentshop/tree/master/recurrentshop https://github.com/nschloe/matplotlib2tikz

• Put your midi data in the folder 'data/original/'

• Group them into folders and name than for example 'style1', 'style2'

• Make sure you have at least 10 midi files per style, otherwise it can't form a test set

• Insert your style names into classes variable in settings.py

• Adjust parameters for training in settings.py

• Make sure you have all these files in the same folder

Training

• Run either vae_training.py to use the full MIDI-VAE model or
• Run any of the style classifiers pitch_classifier.py, velocity_classifer.py or instrument_classifer.py

The models will be stored in the automatically generated folder models/

Evaluation

• Change the model_name and epoch of your MIDI-VAE model that you want to evaluate
• Change the model names and epochs and weights for all the style classifiers
• Make sure you have set the same parameters as were used during training
• Run vae_evaluation.py

midi-vae's Issues

Weird error

Getting this when running vae_training.py (after it imports all the midi files), could you help me please?

.
.
.
Total steps (notes + silent): 2752064
Total samples: 43001
[1.09267808e-01 4.16398747e-01 0.00000000e+00 3.59985585e-01
3.40064972e-01 3.50594625e-01 7.46470084e-03 1.61417231e-02
1.12797230e-04 3.32733292e-03 5.35355538e-03 2.67526337e+00
1.54661519e+00 2.02079281e+00 5.00641917e-01]
[1.31664972e-01 4.16811501e-01 1.00000000e-10 2.99468883e-01
2.83918970e-01 2.91932461e-01 1.08097120e-02 2.48292100e-02
2.10019301e-03 6.62183705e-03 8.46493833e-03 3.14200168e+00
2.29949197e+00 2.51843892e+00 9.06115076e-01]
Training model...
Epoch 0 of 2000 Epochs
Training:
Beta: 0.1
Epsilon std: 0.01
Traceback (most recent call last):
File "vae_training.py", line 809, in
verbose=False)
File "/home/usuario/miniconda3/envs/fastai2/lib/python3.6/site-packages/keras/engine/training.py", line 952, in fit
batch_size=batch_size)
File "/home/usuario/miniconda3/envs/fastai2/lib/python3.6/site-packages/keras/engine/training.py", line 809, in _standardize_user_data
y, self._feed_loss_fns, feed_output_shapes)
File "/home/usuario/miniconda3/envs/fastai2/lib/python3.6/site-packages/keras/engine/training_utils.py", line 273, in check_loss_and_target_compatibility
' while using as loss categorical_crossentropy. '
ValueError: You are passing a target array of shape (5, 1) while using as loss categorical_crossentropy. categorical_crossentropy expects targets to be binary matrices (1s and 0s) of shape (samples, classes). If your targets are integer classes, you can convert them to the expected format via:

from keras.utils import to_categorical
y_binary = to_categorical(y_int)

Alternatively, you can use the loss function sparse_categorical_crossentropy instead, which does expect integer targets.

Versions of TensorFlow and Keras

May I ask what versions of TensorFlow and Keras are used in this model