[11/24/2020] Important future updates will be focused on VoxSRC2020 Tech Reports. Also a large scale ID(1M+ Identities) training scripts will be updated later. More model benchmarks will be updated :)
[10/25/2020] I have released my baseline system ResNet34-AM-VoxCeleb2 based on mmclassification. Thanks to mmclassification, I can fully utilize all the functions and modules provided without paying a lot attention on the training process.
This is a repo intended to provide an open speaker verification tool. Currently this project only provides a training and extraction process. More fundamental functions like feature extraction, post processing, scoring backends and augmentation research will be updated later. The project is based on mmclassification codebase. Please refer to mmclassification readme for installation and running scripts. The code is tested with PyTorch 1.6.0 and CUDA 10.2. NOTE: The pretrained model is saved in PyTorch 1.6.0. So if you are using older versions, you may need to upgrade your PyTorch Version to 1.6.0+ to load our released model.
All features adopted in our training frameworks are extracted from Kaldi, cepstral mean normalized(CMN) and no VAD implemented.
Preparation scripts will be released soon.
- random speaker dataset
- balanced speaker dataset PK sampler
- PK sampler for distributed dataloader Important for metric learning in SV
- Vaex based speaker dataset When your feat.scp file is quite large (e.g. 10M+ lines), this speaker dataset is quite recommended since Vaex format can save both memory and loading time. Instruction are provided to help you transform your kaldi format file to vaex based file. Note: this data set also provide a "pseudo random" window selection sampling strategy as "pseudo".
- dynamic speaker dataset
Instructions to build kaldi format file to vaex format (highly recomended to replace normal speaker dataset with vaex based speaker dataset):
python tools/python/feats2csv.py --feat_scp YOUR_FEAT --utt2spk YOUR_UTT2SPK --out_dir YOUR_DEST --data_type train
Also data_type can be valid or test or whatever you like.
- STATS pooling
- SelfAttention pooling(channel-dependent in ECAPA_TDNN)
- MultiHeadAttention pooling
NOTE: The test set is VOX1-O(cleaned) dataset and training set is VoxCeleb2-dev. Backend is cosine similarity scoring. The minDCF criterion is the same as VoxSRC2020.
Model | Backbone | Metric | feature | batch size | config | raw EER on Vox1-O(cleaned) | raw DCF | raw EER on Vox1-H | checkpoint |
---|---|---|---|---|---|---|---|---|---|
ResNet34-AM-VoxCeleb2 | ResNet34 | AMSoftmax, scale=30, margin=0.2 | 81 FBANK(including energy) | 128 | conf | 1.207 | 0.0738 | 2.44 | ckpt |
ResNet34-AM-VoxCeleb2-syncBN | ResNet34 | AMSoftmax, scale=30, margin=0.2 | 81 FBANK(including energy) | 128 | conf | 1.196 | 0.0791 | - | ckpt |
SEResNet34-AM-VoxCeleb2 | SEResNet34 | AMSoftmax, scale=30, margin=0.2 | 81 FBANK(including energy) | 100 | conf | 1.121 | 0.0771 | 2.43 | ckpt |
SEResNet34-AM-VoxCeleb2-syncBN | SEResNet34 | AMSoftmax, scale=30, margin=0.2 | 81 FBANK(including energy) | 100 | conf | 1.175 | 0.0745 | - | realeased soon |
SEResNet34-AM-VoxCeleb2(with checkpointed) | SEResNet34 | AMSoftmax, scale=30, margin=0.2 | 81 FBANK(including energy) | 128 | conf | 1.07 | 0.0747 | 2.43 | - |