davegabe/ast-ddsp-mss

Novel approach for Music Source Separation using the Audio Spectrogram Transformer for regression on the parameters of Differentiable Digital Signal Processing with additive synthesis.

AST-DDSP: Audio Source Separation with DDSP

This repository contains the code for the project of Deep Learning & Applied AI course at the University of La Sapienza, Rome.

Abstract

When listening to music, we listen to a mixture of different instruments and vocals. Music Source Separation is the task of separating the different sources which compose a music track. In this work a novel approach for MSS is proposed, based on the Audio Spectrogram Transformer performing regression over the parameters of the Differentiable Digital Signal Processing in order to reconstruct the stem track of an instrument from the mixture.

Read the report for more details.

Results

Here we have some example from the testing set of the model trained on the Slakh2100 dataset.

Bass

Track	Mixture	Bass	Bass (AST-DDSP)
1	Download or GitHub Pages	Download or GitHub Pages	Download or GitHub Pages
2	Download or GitHub Pages	Download or GitHub Pages	Download or GitHub Pages
3	Download or GitHub Pages	Download or GitHub Pages	Download or GitHub Pages

Drum

Track	Mixture	Drum	Drum (AST-DDSP)
1	Download or GitHub Pages	Download or GitHub Pages	Download or GitHub Pages
2	Download or GitHub Pages	Download or GitHub Pages	Download or GitHub Pages
3	Download or GitHub Pages	Download or GitHub Pages	Download or GitHub Pages

Acknowledgements

• DDSP
• AST
• Slakh2100
• Auraloss