Giter Site home page Giter Site logo

fzalkow / ctc_chroma Goto Github PK

View Code? Open in Web Editor NEW
4.0 1.0 0.0 15 MB

Repository for the following two papers. F. Zalkow and M. Müller, Using Weakly Aligned Score–Audio Pairs to Train Deep Chroma Models for Cross-Modal Music Retrieval, ISMIR 2020. F. Zalkow and M. Müller, CTC-Based Learning of Chroma Features for Score—Audio Music Retrieval, IEEE/ACM TASLP 2021.

License: Other

Jupyter Notebook 97.55% Python 2.45%

ctc_chroma's Introduction

CTC-Based Learning of Chroma Features for Score—Audio Music Retrieval

Background

This repository contains accompanying code for the following papers. If you use code from this repository, please consider citing them.

[1]: Frank Zalkow and Meinard Müller: Using Weakly Aligned Score–Audio Pairs to Train Deep Chroma Models for Cross-Modal Music Retrieval. In Proceedings of the International Society for Music Information Retrieval Conference, Montréal, Canada, 2020. DOI: 10.5281/zenodo.4245400. [Accompanying website]

[2]: Frank Zalkow and Meinard Müller: CTC-Based Learning of Chroma Features for Score—Audio Music Retrieval. In IEEE/ACM Transactions on Audio, Speech, and Language Processing. 2021. DOI: 10.1109/TASLP.2021.3110137. [Accompanying website]

Usage

You can install the code in this repository with pip:

pip install ctc_chroma

There are two ways to use the models of this repository. The first way is to use a Jupyter notebook. This notebook applies the model and visualizes its output. The second way is to use a script to batch process audio files in a folder. This script can be executed like this:

python apply_model.py -m MODEL_ID -i INPUT -o OUTPUT

Here, INPUT is a directory with audio files, OUTPUT is a directory for the output files, and MODEL_ID specifies the model variant. There are several model variants contained in the repository. These variants are due to different versions of the training data (used in the papers [1] and [2], respectively), due to different training and validation splits, and due to different training procedures. The following table specifies all model identifiers of the repository.

Model Identifier Used in Paper Training Procedure
v1_ctc_train1234valid5 [1] CTC
v1_ctc_train123valid4 [1] CTC
v1_ctc_train2345valid1 [1] CTC
v1_ctc_train234valid5 [1] CTC
v1_ctc_train3451valid2 [1] CTC
v1_ctc_train345valid1 [1] CTC
v1_ctc_train4512valid3 [1] CTC
v1_ctc_train451valid2 [1] CTC
v1_ctc_train5123valid4 [1] CTC
v1_ctc_train512valid3 [1] CTC
v2_ctc_train123valid4 [2] CTC
v2_ctc_train234valid5 [2] CTC
v2_ctc_train345valid1 [2] CTC
v2_ctc_train451valid2 [2] CTC
v2_ctc_train512valid3 [2] CTC
v2_linear_train123valid4 [2] Crossentropy, linear alignment
v2_linear_train234valid5 [2] Crossentropy, linear alignment
v2_linear_train345valid1 [2] Crossentropy, linear alignment
v2_linear_train451valid2 [2] Crossentropy, linear alignment
v2_linear_train512valid3 [2] Crossentropy, linear alignment
v2_strong_train123valid4 [2] Crossentropy, strong alignment
v2_strong_train234valid5 [2] Crossentropy, strong alignment
v2_strong_train345valid1 [2] Crossentropy, strong alignment
v2_strong_train451valid2 [2] Crossentropy, strong alignment
v2_strong_train512valid3 [2] Crossentropy, strong alignment

Recordings

For making it easy to directly try out the code of this repository, we included two excerpts from public domain recordings, which we downloaded from Musopen. The excerpts correspond to the musical sections that are used for the figures in the paper (Figure 3 and 4). However, different performances (not public domain) have been used to generate the figures in the paper. Below you find a small table with details for the excerpts.

Filename Composer Work Performer Description
Beethoven_Op067-01_DavidHighSchool.wav Beethoven Symphony no. 5, op. 67 Davis High School Symphony Orchestra First movement, first theme
Beethoven_Op002-2-01_Pitman.wav Beethoven Piano Sonata no. 2, op. 2 no. 2 Paul Pitman First movement, second theme

Acknowledgements

Frank Zalkow and Meinard Müller are supported by the German Research Foundation (DFG-MU 2686/11-1, MU 2686/12-1). We thank Daniel Stoller for fruitful discussions on the CTC loss, and Michael Krause for proof-reading the manuscript. We also thank Stefan Balke and Vlora Arifi-Müller as well as all students involved in the annotation work, especially Lena Krauß and Quirin Seilbeck. The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institute for Integrated Circuits IIS. The authors gratefully acknowledge the compute resources and support provided by the Erlangen Regional Computing Center (RRZE).

ctc_chroma's People

Contributors

fzalkow avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.