shaoboh/audio-deepdream-tf

Deep dreams on audio spectrograms get resynthesized for deep-learning generated audio effects.

Deep Dreaming on Audio Spectrograms with Tensorflow

This notebook allows imagenet to hallucinate audio effects onto an mp3.

It uses the tensorflow primer on DeepDreaming to adapt Christian Dittmar & Stefan Balke's DeepDreamEffect for Caffe from HAMR 2015 to tensorflow. In addition, the loss function for inducing hallucinations in the convnet is edited such that high energy areas of the spectrogram are ignored, thereby avoiding distortion upon resynthesis and adding musicality to the effect.

In essence, this hack converts audio spectrograms into images, where they can be processed by specific layers of a pre-trained convulutional neural network (Inception v3 trained on ImageNet) , and then re-synthesized into audio.

Dependencies:

Python 3.5.x

TensorFlow 0.11 for deep learning.

Librosa 0.4.3 for audio DSP.

numpy 1.11.1