qcoleman97/audiotagger

Deep Learning Neural Networks Final Project

AudioTagger

See the Write-Up

Based on Kaggle's Freesound General-Purpose Audio Tagging Challenge from 2018: https://www.kaggle.com/c/freesound-audio-tagging/

This is a model composition that infers the source of a sound (trumpet, meow, etc.) given audio input, using the Keras library in Python. TagAudio.ipynb is the golden notebook.

audiotagger's Issues

Compare with baseline model

Baseline model provided by DCASE 2018:
https://github.com/DCASE-REPO/dcase2018_baseline/tree/master/task2

DCASE 2018: http://dcase.community/challenge2018/task-general-purpose-audio-tagging

Create Model

2D CNN's promising via Kaggle.
1 on leaderboard: http://dcase.community/documents/challenge2018/technical_reports/DCASE2018_Jeong_102.pdf
2 on leaderboard: https://cpjku.github.io/dcase_task2/
3 on leaderboard: http://dcase.community/documents/challenge2018/technical_reports/DCASE2018_Iqbal_89.pdf

Ensemble of top methods?
https://www.kaggle.com/daisukelab/freesound-dataset-kaggle-2018-solution

ch 7: (advanced prac)
model ensembling (of 2D CNN & Combined 1D CNN & RNN combo, etc??)
Replace Conv2D w/ DepthwiseConv2D (or maybe SeperableConv2D?)
Batch normailization
Inception
Residual connections
DenseNet?

ch 6: (Sequential methods)
Staley - 1D CNN is preprocessing step before RNN
. Bidirectional RNNs, recurrent dropout, & stacking RNNS

ch 5: (Convolution)
Tune HP (# neurons, layers, epochs, batch_size)
. Dropout, regularization
. Data augmentation - No
. Use a pretrained CNN? (VGG16)
. Fine-tuning (VGG16)

K-Fold Validation

Import Train & Test Data w/ corresponding labels

Pre-process data

Things to possibly do in order:
Cut out silent parts
Normalize wave form
Do STFT - transforms them from waveforms to spectrograms (STFT) to log mel spectrograms (1)
sample shape (timeframes, frequencies, 1)
Possibly normalize again

ExampleSTFT.py is resource, and can comment out the pyaudio (just for playing WAV files)

(1) Converting spectrograms to log mel spectrograms, or MFCC, could be even better
From wikipedia - "MFCCs are also increasingly finding uses in music information retrieval applications such as genre classification, audio similarity measures, etc.[7]"

Not high priority: copy spectrogram samples converted into shape (timeframes, frequencies) to be fed into 1D Convnets