ws-choi/lasaft-net-v2

A PyTorch implementation: "LASAFT-Net-v2: Listen, Attend and Separate by Attentively aggregating Frequency Transformation"

LASAFT-Net-v2

Listen, Attend and Separate by Attentively aggregating Frequency Transformation

Woosung Choi, Yeong-Seok Jeong, Jinsung Kim, Jaehwa Chung, Soonyoung Jung, and Joshua D. Reiss

Demonstration (under construction)

Experimental Results

• Musdb 18

model	vocals	drums	bass	other	AVG
Meta-TasNet	6.40	5.91	5.58	4.19	5.52
AMSS-Net	6.78	5.92	5.10	4.51	5.58
LaSAFT-Net-v1	7.33	5.68	5.63	4.87	5.88
LASAFT-Net-v2	7.57	6.13	5.28	4.87	5.96

• MDX Challenge (Leaderboard A)

model	model type	vocals	drums	bass	other	AVG
KUILAB-MDX-Net	dedicated (1 source/ 1 model)	8.901	7.173	7.232	5.636	7.236
LaSAFT-Net-v1 (light)	conditioned (4 sources/ 1 model)	7.275	5.935	5.823	4.557	5.897
LASAFT-Net-v2 (light)	conditioned (4 sources/ 1 model)	7.324	5.976	5.884	4.642	5.957

How to reproduce

1. Environment

• Ubuntu 20.04
• wandb for logging

You must create .env file by copying .env.sample to set environmental variables.

wandb_api_key=[Your Key] # "xxxxxxxxxxxxxxxxxxxxxxxx"
data_dir=[Your Path] # "/home/ielab/repos/musdbHQ"

• about wandb_api_key
  • we currently only support wandb for logging.
  • for wandb_api_key, visit wandb, go to setting, and then copy your api key
• about data_dir
  • the absolute path where datasets are stored

2. Installation (cuda)

conda env create -f environment.yaml -n lasaftv2
conda activate lasaftv2
pip install -r requirements.txt

lasaft-net-v2's Issues

without skip connections, and less bn

easily saturated or NaN

WHICH LaSAFT V2 IS THE BIG MODEL OR THE LIGHT VERSION?

I noticed that in confi_path we are informed that it is the model "lasaft/pretrained/v2_light" is that correct? it would be possible to confirm, and if so, it would be possible to add the larger model, as the light version gives us inferior results compared to the big version, at least that's what I saw when I tested the V1 version of LaSAFT, thanks again!

CUDA (GPU) IS NOT BEING RECOGNIZED

Hello, how are you W. Choi?

Remember that I asked you several months ago how to use cuda instead of cpu in LaSAFT v1? so I would like to ask you how to make cuda recognized in LaSAFT v2, remembering that I already created a conda env as informed and installed everything correctly, but even so it is only recognizing the cpu instead of the gpu,

I await your return,
Sincerely, Lucas Rodrigues.