jamesliao2016/transfer_learning_music

Transfer learning for music classification and regression tasks

transfer_learning_music

Repo for paper "Transfer learning for music classification and regression tasks" by Keunwoo Choi et al.

Mode 1/2. To use the pre-trained convnet feature extractor

For your own music/audio-related work.

Prerequisites (Same as mode 2 except datasets)

• Theano. I used version 0.9.0 but should work with some similar versions.
• Keras 1.2.2 (OLD ONE!) (NOT THE MOST RECENT VERSION)
  • set image_dim_ordering : th in ~/keras/keras.json
  • set backend : theano, too.
• Kapre OLD VERSION for OLD KERAS In short,

$ pip install theano==0.9
$ pip install keras==1.2.2
$ git clone https://github.com/keunwoochoi/kapre.git
$ cd kapre
$ git checkout a3bde3e
$ python setup.py install

Usage

$ python easy_feature_extraction.py audio_paths.txt some/path/features.npy

where audio_path.txt is line-by-line audio paths and some/path/features.npy is the path to save the result.

E.g., audio_path.txt :

blah/a.mp3
blahblah/234.wav
some/other.c.mp3

Then load the .npy file. The features are size of (num_songs, 160).

Mode 2/2. To reproduce the paper

Prerequisites

• Download datasets:

  • Extended ballroom
  • Gtzan genre
  • Gtzan speech/music
  • emoMusic
  • Jamendo singing voice
  • Urbansound8K

• Prerequisites

  • Theano. I used version 0.9.0 but should work with some similar versions.
  • Keras 1.2.2 (OLD ONE!) (NOT THE MOST RECENT VERSION)
    • set image_dim_ordering : th in ~/keras/keras.json
    • set backend : theano, too.
  • Kapre OLD VERSION for OLD KERAS by

$ git clone https://github.com/keunwoochoi/kapre.git
$ cd kapre
$ git checkout a3bde3e
$ python setup.py install

• Optionally, Sckikt learn, Pandas, Numpy,.. for your convenience.

Usage

• 0. main_prepare_many_datasets.ipynb: prepare dataset, pre-processing
• 1. feature extraction for 6 tasks.ipynb: feature extraction (MFCC and convnet features)
• 2_main_knn_svm_transfer: Do SVM
• 3. knn and svm (with AveragePooling) results plots: Plot results

Appendix

Links

• Train/valid/test split of MSD that I used for the training
• Paper: arXiv 1703.09179, Transfer Learning for Music Classification and Regression tasks
• Blog article

Citation:

@inproceedings{choi2017transfer,
  title={Transfer learning for music classification and regression tasks},
  author={Choi, Keunwoo and Fazekas, George and Sandler, Mark and Cho, Kyunghyun},
  booktitle={The 18th International Society of Music Information Retrieval (ISMIR) Conference 2017, Suzhou, China},
  year={2017},
  organization={International Society of Music Information Retrieval}
}