tandav/pitch-detectors

collection of pitch (f0, fundamental frequency) detection algorithms with unified interface

pitch-detectors

collection of pitch (f0, fundamental frequency) detection algorithms with unified interface

list of algorithms

algorithm	cpu	gpu	MDBStemSynth accuracy	Mir1K accuracy
Crepe	✓	✓	0.886 ± 0.059	0.759 ± 0.073
Penn	✓	✓	0.699 ± 0.263	0.660 ± 0.090
PipTrack	✓		0.524 ± 0.286	0.864 ± 0.071
PraatAC	✓		0.777 ± 0.302	0.859 ± 0.074
PraatCC	✓		0.776 ± 0.300	0.872 ± 0.068
PraatSHS	✓		0.534 ± 0.238	0.578 ± 0.169
Pyin	✓		0.722 ± 0.252	0.888 ± 0.047
Rapt	✓		0.791 ± 0.282	0.827 ± 0.078
Reaper	✓		0.792 ± 0.261	0.795 ± 0.083
Spice	✓	✓	0.024 ± 0.034	0.889 ± 0.046
Swipe	✓		0.784 ± 0.276	0.796 ± 0.062
TorchCrepe	✓	✓	0.764 ± 0.273	0.774 ± 0.084
TorchYin	✓		0.735 ± 0.268	0.866 ± 0.052
World	✓		0.869 ± 0.135	0.842 ± 0.075
Yaapt	✓		0.740 ± 0.271	0.744 ± 0.107
Yin	✓		0.749 ± 0.269	0.884 ± 0.043

accuracy is mean raw pitch accuracy

install

pip install pitch-detectors

All agorithms tested on python3.12, this is recommended python version to use

usage

from scipy.io import wavfile
from pitch_detectors import algorithms
import matplotlib.pyplot as plt

fs, a = wavfile.read('data/b1a5da49d564a7341e7e1327aa3f229a.wav')
pitch = algorithms.Crepe(a, fs)
plt.plot(pitch.t, pitch.f0)
plt.show()

Colab notebook with plots for all algorithms/models

additional features

• robust (vote + median) ensemble algorithm using all models
• json import/export

notes:

Tests are running in subprocess (using scripts/run_algorithm.py) to avoid pytorch cuda import caching. It's difficult to disable gpu after it has been initialized. (pytorch/pytorch#9158) It is also difficult to set correct PATH and LD_LIBRARY_PATH without a subprocess.

pitch-detectors's Issues

try add parselmouth to_pitch_spinet

https://parselmouth.readthedocs.io/en/stable/api_reference.html#parselmouth.Sound.to_pitch_spinet

add benchmark gpu vs cpu time for gpu-based algorithms

add 3 second and 30 seconds files to benchmark

Colab script

Can you please share a colab notebook with all the algorithms and plots for a file?

I am facing an issue with

• penn
• torchcrepe

add benchmarks on some dataset

Add median to comparison

The MIR-1k [17] dataset contains
1000 audio tracks with people singing Chinese pop songs. The
dataset is annotated with pitch at a granularity of 10 ms and
it also contains voiced/unvoiced frame annotations. It comes
with two stereo channels representing, respectively, the singing
voice and the accompaniment music.

pitch ground truth labels in PitchLabel (.pv files) are midi-note values, not hz. You have to convert it to get f0

Use right channel in Wavfile folder

• mir-1k dataset
• MIR-1K dataset | Zenodo
• https://www.kaggle.com/datasets/datongmuyuyi/mir1k
• crepe paper
• spice paper
• mir_eval paper
• craffel/mir_eval: Evaluation functions for music/audio information retrieval/signal processing algorithms.
• Code search results · GitHub
• Code search results · GitHub

try add librosa.yin (in addition to pyin)

Docker stucks because of absence of -i flag in docker run command

test PITCH_DETECTORS_GPU=true errors if GPU not available for new subclass scheme

.PHONY: test-no-gpu
test-no-gpu: build
	docker run --rm -t \
	-e PITCH_DETECTORS_GPU=true \
	$(IMAGE) \
	pytest -v --cov pitch_detectors

use unified interface for median-vote averaging algorithm

try to support passing all algo-specific **kwargs

change ceiling up to 1500 for benchmarks

add note about how to run in docker

align pitches as in mir-1k dataset (t 10ms, 20ms, 30ms, etc)

fix return_periodicity torchcrepe warning

bastibe/Replication-Dataset-Scripts: Scripts for recreating the Replication Dataset for Fundamental Frequency Estimation. Part of the dissertation "Pitch of Voiced Speech in the Short-Time Fourier Transform".

• bastibe/Replication-Dataset-Scripts: Scripts for recreating the Replication Dataset for Fundamental Frequency Estimation. Part of the dissertation "Pitch of Voiced Speech in the Short-Time Fourier Transform". © 2020, Bastian Bechtold. All rights reserved.

Use table for list of algorithms

run ci in nvidia docker image

try add viterbi to piptrack

https://chat.openai.com/chat/b24e346a-df2b-4e2a-bb26-0ea50ef55643

run accuracy computation in CI

add pitch at time=0 to f0-true

fix LD_LIBRARY_PATH to run without docker

• test CPU as well (gpu must not be available)
• fix hardcoded LD_LIBRARY_PATH it should be system independent (and run in CI)
• fix hardcoded model paths

add standard deviation to metrics

replace scipy wavefile io with librosa

UPD: nope, because librosa have scipy as dependency

sort algorithms alphabetically in markdown table

dont fail by default if gpu is not available. Add option to fail for algorithms with GPU support

convert datasets to single standard format (use mir_eval 2 column csv)

delete pytest-order

add DIO from pyworld

https://github.com/interactiveaudiolab/penn/blob/962af14f29947b33030223d6d8637ab2b475ff24/penn/dsp/dio.py#L15

add links to algorithms github or docs homepages

Add other datasets

from crepe:
MIR-1K [1]
Bach10 [2]
RWC-Synth [3]
MedleyDB [4]
MDB-STEM-Synth [5]
NSynth [6]

• https://mirdata.readthedocs.io/en/0.3.0/source/quick_reference.html
• marl/crepe#39
• https://hal.science/hal-01161386/document
• http://synthdatasets.weebly.com/
• https://www.spsc.tugraz.at/databases-and-tools/ptdb-tug-pitch-tracking-database-from-graz-university-of-technology.html

try different ensemble algorithms to maximize quality

• try vote based on quality of algorithms (weight)
• https://scikit-learn.org/stable/modules/ensemble.html

review alignment, probably it's wrong for some algorithms (crepe quality is looks lower than it should be)

• try add pitch at time=0 to f0-true #37
• mdb-stem-synth dataset has ground truth t for pitch values. Try use it to adjust time in algorithms
• try to adjust thresholds for crepe/torchcrepe
• https://chat.openai.com/chat/dfb00e2a-8ea3-4edd-acfd-f89f23a1826c

numba cuda.get_current_device() does not help to prevent GPU memory leak

Current solution:

device = cuda.get_current_device()
device.reset()

1. Try to run in subprocess. Or run each algorithm in separate docker run command
2. Try use tensorflow growth parameter

add https://github.com/interactiveaudiolab/penn

test coverage

make a service with api and web interface. Upload/record audiofile and get back videos, f0-json, images etc. Use queues etc

• json can takes either .wav binary blob or PCM as json array

deepf0

• Code search results · GitHub

add docstrings and docs

review Swipe algorithm. Seems like it does not have gpu support

configure GPU memory limit using environment variable

d414904

fix pytest warnings (tensorflow)

make python script to generate markdown table

ensemble algorithm should support creation from generic F0 schemas

Add deepF0 algo https://arxiv.org/pdf/2102.06306.pdf

https://arxiv.org/pdf/2102.06306.pdf
https://sigport.org/sites/default/files/docs/ICASSP_2021_Poster-final-version.pdf

UPD: there's no open source implementation, so closing the issue

add PEFAC algorithm

https://www.google.com/search?q=PEFAC+pitch+estimation+python&newwindow=1&hl=en&ei=39YeZNS6EZK-sAfZ-bGwCw&ved=0ahUKEwiUpsKR-fb9AhUSH-wKHdl8DLYQ4dUDCA8&uact=5&oq=PEFAC+pitch+estimation+python&gs_lcp=Cgxnd3Mtd2l6LXNlcnAQAzIFCCEQoAE6CwgAEKIEELADEIsDOgcIIRCgARAKSgQIQRgBUOgCWLQMYNEPaAFwAHgCgAGuBYgBvRGSAQswLjEuMS4wLjEuMpgBAKABAcgBAbgBAsABAQ&sclient=gws-wiz-serp

add https://github.com/alexanderlerch/pyACA/tree/master

https://github.com/alexanderlerch/pyACA/tree/master

Use base bytes instead of base digest in source_hashes

split algorithms into multiple files. Check file sha256 before running. If not changed - dont run and used cached value of accuracy