juliamusic/pianofingering.jl

(Experimental, a lot of bugs) Advanced automatic fingering generator for piano scores, determining optimal fingering using Model-Based Reinforcement Learning, written in the Julia language.

PianoFingering.jl

PianoFingering.jl is an automatic fingering generator for piano scores, written in the Julia language.

There is no "best fingering", the fingerings generated by PianoFingering.jl are for reference only.

THIS PACKAGE ONLY TESTED ON LINUX.

Future Works:

1. Support input of recorded MIDI. (This requires training neural networks to split notes for the left hand and right hand.)

2. Replace the annotation algorithm to combine data-driven and rule-based methods.

3. Remove the music21. (Maintain MusicXML.jl)

How to use it

Pre-requirement

Prompt: You can copy all following commands start with ❯

Install Python (>3.11) and Julia (>1.9).

Make sure:

❯ python -V
Python 3.11.3
❯ julia -v
julia version 1.9.0

Install music21 (>9.1.0):

❯ pip3 install music21

Clone this project to your computer and change directory to PianoFingering.jl folder, then:

❯ julia
❯ using Pkg
❯ Pkg.activate(".")
❯ Pkg.instantiate()

Now you have installed all the dependency. Close your terminal.

Run Programe

1. Put your .musicxml file to musicxml folder, suppose we have an example.musicxml file right now.

2. Change directory to PianoFingering.jl folder:

❯ julia -t auto
❯ using Pkg
❯ Pkg.activate(".")
❯ using PianoFingering
❯ fingering("example.musicxml")

Wait for the program to complete, annotated piano score example_output.musicxml will be stored in output folder, this may take a few minutes.

PIG Dataset

PIG is publicly available data for academic research on piano fingering.

If you want to fingering a PIG format .txt file, just put .txt in pig folder. Then after using PianoFingering:

❯ fingering("pig_example.txt")

Musicxml to PIG

Fingering a .musicxml file to PIG format:

❯ xml_to_pig("example.musicxml")

Feedback

If you encounter error, or musicxml file can't fingering correctly. Please open a issue and upload your musicxml file.

pianofingering.jl's Issues

Musescore 4 .mscz fingering output support.

The biggest problem is there is no Julia mscz file parser.

We need to implement one or use the python ms3 library.

Implement Rule-based and Data-driven fingering algorithms.

For now, PianoFingering.jl uses pure rule-based fingering algorithms. It is based on simulated piano performance, use reinforcement learning to minimal motion.

Advantages of rule-based fingering:

• Personalised fingering, different fingering output for different hand size.
• Physical playable, it's based on a simulated play.

Advantage of data-driven fingering:

• fluent, learning from human annotated music score.

The challenge is to combine these two different ways. My next step is to add pure data-driven fingering algorithms.

Then, hard constraints will be added to the data-driven model for personalize fingering output.

A lot of type instablities

I am not joking.
I recommend you to use my SimpleTypeChecker.jl to scan your files, which can capture typing error more easily.

1. Declare global variables in const.jl as const
white_keys, black_keys should be declared with const xxx = .... Otherwise they are inferred as Any because they can be assigned with arbitrary values. This impacts a lot of functions.
2. Reduce unnecessary use of .... Mostly of time, there is no necessary to convert an iterator to a Vector, because it supports iterator protocol.
3. findfirst at const.jl:335 return Union{Nothing, ...}. All find functions return Union..., it's a good habit to check nothing and prevent usage of a Union{Nothing}
4. In const.jl, const Fingering = SortedDict{Note, Finger} is an abstract type.
   Defined it as const Fingering = SortedDict{Note, Finger, Base.Order.ForwardOrdering}
5. In split.jl:77, split_length = ceil(notes_length / threads_num), ceil returns a Floating Number. You need to cast it to a Integer. Because it's reassigned with an Integer in the following code.

If you are unsure of what I have just said, please commit in this issue.

musicxml error

My musicxml file is running reported error.
1.musicxml.zip

end: part:9 length:33
On Iteration 200, Returns: 3414.3837769328175
On Iteration 200, Returns: 3609.6895578828744
On Iteration 400, Returns: 3418.8118533184743
On Iteration 400, Returns: 3609.837874984363
On Iteration 600, Returns: 3418.8118533184743

end: part:5 length:81
On Iteration 600, Returns: 3609.880186651809

On Iteration 800, Returns: 3609.880186651809
end: part:10 length:85
52.382566 seconds (811.76 M allocations: 30.790 GiB, 18.85% gc time, 102.55% compilation time)
ERROR: number of fingering and note not equal.
Stacktrace:
[1] error(s::String)
@ Base ./error.jl:35
[2] annotation(part::PyCall.PyObject, fgs::Vector{DataStructures.SortedDict{MIDI.Note, PianoFingering.Finger}})

Error

Hello, I'm using Windows Subsystem for Linux and got an error after uploading my musicxml file. Attached are two screenshots of the error. Any help would be appreciated, looking forward to using this. Thanks!

Bryan

ERROR: KeyError: key :flatten not found

I get this error
ERROR: KeyError: key :flatten not found
Stacktrace:
[1] __getproperty
@ ....julia\packages\PyCall\twYvK\src\PyCall.jl:313 [inlined]
[2] getproperty
@ ....julia\packages\PyCall\twYvK\src\PyCall.jl:318 [inlined]
[3] annotation(part::PyCall.PyObject, fgs::Vector{DataStructures.SortedDict{MIDI.Note, PianoFingering.Finger}})
@ PianoFingering .....PianoFingering.jl\src\process.jl:78
[4] fingering(file_name::String)
@ PianoFingering .....PianoFingering.jl\src\process.jl:23
[5] top-level scope
@ REPL[4]:1

Remove music21 dependency.

For now, we use music21 python library to parse and write musicxml files.

MusicXML.jl is no longer maintained, maybe this work will be better after the library finds a new maintainer.

Roadmap

• PIG dataset loader
• midi to PIG without/with fingering
• musicxml to PIG without/with fingering
• support repeat measure
• more reward function improve

Neural network to separate left and right hand notes.

To support performance midi input, we need to train a neural network for separate left hand and right hand notes.
The potential solution is RNN, LSTM or bi-LSTM.

If anyone wants to help, please comment freely, leave your thoughts.