apmcleod/harmonic-inference

harmonic-inference's Issues

Improve annotation speed

Optimizing the beam search is an option, as well as running some of the models on GPU.

Write a better test for score_piece

Now that repeats are removed, weird things happen when key changes interrupt identical chords.

annotate.py does not run with newer versions of torchmetrics and setuptools

Tried python annotate.py -h after doing the indicated installation in a python=3.9.5 environment which left me with torch==1.9.0 and torchmetrics==0.11. The result was

ImportError: cannot import name 'get_num_classes' from 'torchmetrics.utilities.data' (.../lib/python3.9/site-packages/torchmetrics/utilities/data.py)

Downgrading to torchmetrics==0.7 solved the issue as per pesser/stable-diffusion#13

Then the same command yielded

AttributeError: module 'distutils' has no attribute 'version'

which I managed to solve by installing setuptools==59.5.0 as per https://stackoverflow.com/questions/74684612/attributeerror-module-distutils-has-no-attribute-version

Implement Chord Sequence reduction mapping for pieces

Sometimes, consecutive chords are the same, just with different inversions. In these cases, we want to allow the user to merge those into a single chord symbol (at least, with no change between the chords).

Have some idea of relative pitch height

Relative to pitches of notes within some window of a chord, maybe 2 bars, maybe the chord duration?

The model should find "low" notes, even if they are only low locally, but not globally.

CSM to use different output representation

Maybe bag of pitches? Voice-based? bass and pitches?

This is possible since it's output is only used as a prior, not as output.

Check onset_level cache

During the beam search, this seems to be giving incorrect results.

get_inputs_within_range misses some notes.

For example, when one note is held through the chord, but other non-chord notes are between that note and the chord onset.

Write code to add reduction to h5 data on load

The reduction should be able to be performed post creation for each dataset.

Finish tests for get_notes_within_chord

Remove unused chords/notes after removing repeats

min_pitch and max_pitch are incorrect!

The minimum (octave, TPC) tuple is not necessarily the minimum pitch (and same for maximum)

Log merge_notes better

Specifically, log any final unmatched tied_in_notes and any tied notes beginning with tied == 0.

Test ScorePiece creation

More flexible and complete type checking/conversion in joint model.

Missing tests for 7th chords in get_chord_type_string()

Specific Places to check

Mozart K_279-2 (index 566), bar 46 has a C# chord. This is presumably being driven by the CSM, although the following key change makes no sense. This is also enharmonic to the correct chord (there is a Db)...:

Index = 712

Add chord_type set reduction maps

ie, if someone wants to only use MAJOR, MINOR, AUGMENTED, DIMINISHED, each additional chord should map down to it. Perhaps also add an "OTHER" base chord type?

Read ICM kwargs in test.py

To use reductions, and remove inversions in ICM, this can be useful.

Off-by-one error

There is an off-by-one or off-by-margin error, particularly for the one-hot key/chord relative calculation in State.

Allow to load reduction, other enums from json

Maybe as string "MAJOR", or int, e.g.: ChordType(int)

CCM transposition invariant.

Can return None

It is theoretically possible to go through a piece and finish with an empty Beam. What to do in such cases?

Handle Augmented 6 chords

Per conversation with Johanes, probably best would be to add a new "Aug6" class with all of them, with the root at scale degree 2, #4, or b6.

Currently, they are just minor on 6 (or b6).

Installation works with Python 3.9.5 exclusively

At first I tried to install in a Python 3.10 environment (which is also what you get when creating the conda environment with python=3) but installation failed while a C subroutine was compiling numpy. Then I used an environment created with python=3.9 which lead to

INFO: pip is looking at multiple versions of harmonic-inference[dev] to determine which version is compatible with other requirements. This could take a while.
ERROR: Package 'harmonic-inference' requires a different Python: 3.9.16 not in '==3.9.5'

In a new environment created with python=3.9.5 the installation worked.

With onsets_only, some chords have 0 notes.

Decide how to handle these. (Probably by not ignoring tied notes). Another option: exclude chords with no notes.

`utils.remove_repeats()` eats up the measures table

Trying out annotating with forced labels, first attempt with a single piece. Here's the aggregated data: 146_wer.zip

Edit: It seems to be happening only in this special case where dealing with a single piece.

Steps to reproduce

Extract to corpus_data and run

python annotate.py -i ../corpus_data/ -o output --label-type dcml --checkpoint checkpoints-beat-best -l annotate.log

Error

Occurred with the current 1.0.0 version on the output branch:

Traceback (most recent call last):
  File "~/harmonic-inference/annotate.py", line 376, in <module>
    pieces = load_pieces(xml=ARGS.xml, input_path=ARGS.input, specific_id=ARGS.id)
  File "~/harmonic-inference/harmonic_inference/utils/data_utils.py", line 125, in load_pieces
    files_df, measures_df, chords_df, notes_df = load_clean_corpus_dfs(input_path)
  File "~/harmonic-inference/harmonic_inference/data/corpus_reading.py", line 202, in load_clean_corpus_dfs
    notes_df = cu.add_note_offsets(notes_df, measures_df)
  File "~/harmonic-inference/harmonic_inference/utils/corpus_utils.py", line 291, in add_note_offsets
    if isinstance(measures.iloc[0]["next"], list) or isinstance(measures.iloc[0]["next"], tuple):
  File "~/miniconda3/envs/harmony/lib/python3.10/site-packages/pandas/core/indexing.py", line 1073, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "~/miniconda3/envs/harmony/lib/python3.10/site-packages/pandas/core/indexing.py", line 1625, in _getitem_axis
    self._validate_integer(key, axis)
  File "~/miniconda3/envs/harmony/lib/python3.10/site-packages/pandas/core/indexing.py", line 1557, in _validate_integer
    raise IndexError("single positional indexer is out-of-bounds")
IndexError: single positional indexer is out-of-bounds

Origin

I could trace back the error to corpus_reading.py#L140. After this call, measures_df is empty.

`annotate.py` fails in the presence of `chords.tsv`

Calling python annotate.py -i corpus_data -o corpus_data --label-type dcml --checkpoint checkpoints-beat-best -l annotate.log on this corpus_data.zip works only if one removes chord.tsv, otherwise it fails with

Traceback (most recent call last):
  File "~/harmonic-inference/annotate.py", line 376, in <module>
    pieces = load_pieces(xml=ARGS.xml, input_path=ARGS.input, specific_id=ARGS.id)
  File "~/harmonic-inference/harmonic_inference/utils/data_utils.py", line 169, in load_pieces
    pieces = [
  File "~/harmonic-inference/harmonic_inference/utils/data_utils.py", line 170, in <listcomp>
    get_score_piece_from_data_frames(
  File "~/harmonic-inference/harmonic_inference/data/piece.py", line 970, in get_score_piece_from_data_frames
    non_repeated_mask = get_reduction_mask(
  File "~/harmonic-inference/harmonic_inference/data/piece.py", line 75, in get_reduction_mask
    mask = np.full(len(inputs), True, dtype=bool)
TypeError: len() of unsized object

NB: chords.tsv contains exactly one chord label per piece.

Duration_cache is calculated 2x per piece.

Pass it through from joint_model to get_note_vectors

Loading forces fails with `AttributeError: 'NoneType' object has no attribute 'get_score_piece'`

In order to reproduce this bug, checkout #76 to fix an ms3 bug in the discovery of scores containing forces.

The error message occurs for this corpus_data.zip when executing python annotate.py --label-type dcml --checkpoint checkpoints-beat-best -l annotate.log -i ~/corpus_data -o ~/corpus_data --forces ~/corpus_data/MS3.

Getting harmony for pieces:   0%|                                                                       | 0/5 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "~/harmonic-inference/annotate.py", line 378, in <module>
    annotate(
  File "~/harmonic-inference/annotate.py", line 185, in annotate
    state, estimated_piece = model.get_harmony(piece, **forces_dict)
  File "~/harmonic-inference/harmonic_inference/models/joint_model.py", line 789, in get_harmony
    self.kppm_post_processing(state)
  File "~/harmonic-inference/harmonic_inference/models/joint_model.py", line 1605, in kppm_post_processing
    piece = state.get_score_piece(
AttributeError: 'NoneType' object has no attribute 'get_score_piece'

Check if ksm return is equal to previous key is wrong

Currently, it's priors_argosrt[0] == state.key, but ksm returns are relative indexes, while state.key is absolute.

Implement beam search for ChordTransitionModel

Use pitch_types

Convert to pitch_types library, for octave, MIDI -> TPC conversion, and min/max pitch.

Fix pandas column type warning in read_dump

DtypeWarning: Columns (18,19) have mixed types.Specify dtype option on import or set low_memory=False.

This is in regards to *_is_minor columns, which are nullable boolean-type.

Invalid chords in beam search

The full range of relative chords is not always valid in absolute terms, similar to the key_change input vector. Think about whether this is an issue.

Add "distance to next_onset", "distance_from_prev_onset" to note vectors?

CTM may need these...

Stratify training/testing by composer sometimes.

Interpret joint_model outputs using dataset_kwargs

eg, to have different inversions usable, in some cases.

Check "bass_note" meaning

Presumably, this should be calculatable from root, chord_type, and inversion, but it isn't always correct.

Maybe it actually means "lowest note in the chord"?

Add annotate.py to run on a non-annotated score.

Should allow for both musicxml and musescore3 (and DCML tsv format).

Allow for locking in certain labels

For example: "note 7 should be C:maj"

More testing

• rhythmic_utils functions that require measures_df need testing.
• corpus_utils needs testing
• harmonic_inference_data needs testing

Re-do and test merge_notes

Error in chords

validation piece 1, chords 3 and 4 are the same:

pieces[1].get_chords()[3:5]:

Chord(root=21, bass=18, key_tonic=19, key_mode=KeyMode.MINOR, chord_type=ChordType.DIMINISHED, inversion=1, onset=(3, Fraction(3, 8)), onset_level=2, offset=(3, Fraction(5, 8)), offset_level=1, duration=1/4, pitch_type=PitchType.TPC),
       Chord(root=21, bass=18, key_tonic=19, key_mode=KeyMode.MINOR, chord_type=ChordType.DIMINISHED, inversion=1, onset=(3, Fraction(5, 8)), onset_level=1, offset=(4, Fraction(0, 1)), offset_level=3, duration=1/8, pitch_type=PitchType.TPC)

Add duration information to chord vectors

Currently, duration information is not included in a chord vector.

Re-write get_offsets, remove_repeats, and merge_ties with new data

Perhaps take some from mozart_loader.

Merge consecutive chord ranges with the same chord

At inference time

Unable to process an input with single measure

Running the pretrained model on an mxl file with a single measure results in the following crash:

Loading pieces:   0%|                                                                             | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
  File "/home/napulen/harmonic-inference/annotate.py", line 327, in <module>
    pieces = load_pieces(xml=ARGS.xml, input_path=ARGS.input, specific_id=ARGS.id)
  File "/home/napulen/harmonic-inference/harmonic_inference/utils/data_utils.py", line 152, in load_pieces
    pieces = [
  File "/home/napulen/harmonic-inference/harmonic_inference/utils/data_utils.py", line 153, in <listcomp>
    get_score_piece_from_music_xml(
  File "/home/napulen/harmonic-inference/harmonic_inference/data/piece.py", line 943, in get_score_piece_from_music_xml
    measures_df = get_measures_df_from_music21_score(m21_score)
  File "/home/napulen/harmonic-inference/harmonic_inference/data/piece.py", line 779, in get_measures_df_from_music21_score
    ts_epoch = Fraction(list(m21_score.measureOffsetMap().keys())[1] / 4)
IndexError: list index out of range
Loading pieces:   0%|                                                                             | 0/1 [00:00<?, ?it/s]

Can confirm that the same input with an additional measure that repeats the last chord was successfully processed.

Allow to load dataset reduction, inversion, etc from model definition.

Since model args are already parsed from command line, this is the easiest way.

For example, could add a model.get_dataset_kwargs(self) -> Dict[str, Any] function.

Check get_range_length with new offset representation.