twistedmove/open-speaker-verification

speaker verification tool

Open Speaker Verification

News

[11/24/2020] Important future updates will be focused on VoxSRC2020 Tech Reports. Also a large scale ID(1M+ Identities) training scripts will be updated later. More model benchmarks will be updated :)

[10/25/2020] I have released my baseline system ResNet34-AM-VoxCeleb2 based on mmclassification. Thanks to mmclassification, I can fully utilize all the functions and modules provided without paying a lot attention on the training process.

Introduction

This is a repo intended to provide an open speaker verification tool. Currently this project only provides a training and extraction process. More fundamental functions like feature extraction, post processing, scoring backends and augmentation research will be updated later. The project is based on mmclassification codebase. Please refer to mmclassification readme for installation and running scripts. The code is tested with PyTorch 1.6.0 and CUDA 10.2. NOTE: The pretrained model is saved in PyTorch 1.6.0. So if you are using older versions, you may need to upgrade your PyTorch Version to 1.6.0+ to load our released model.

Data preparation

All features adopted in our training frameworks are extracted from Kaldi, cepstral mean normalized(CMN) and no VAD implemented.

Preparation scripts will be released soon.

Attribute

dataset

• random speaker dataset
• balanced speaker dataset PK sampler
• PK sampler for distributed dataloader Important for metric learning in SV
• Vaex based speaker dataset When your feat.scp file is quite large (e.g. 10M+ lines), this speaker dataset is quite recommended since Vaex format can save both memory and loading time. Instruction are provided to help you transform your kaldi format file to vaex based file. Note: this data set also provide a "pseudo random" window selection sampling strategy as "pseudo".
• dynamic speaker dataset

Instructions to build kaldi format file to vaex format (highly recomended to replace normal speaker dataset with vaex based speaker dataset):

python tools/python/feats2csv.py --feat_scp YOUR_FEAT --utt2spk YOUR_UTT2SPK --out_dir YOUR_DEST --data_type train

Also data_type can be valid or test or whatever you like.

pipeline

• SpecCutout

backbone

• ResNet34
• SEResNet34
• SEResNet34-MSEA
• RES-SE-TDNN

Pooling Method

• STATS pooling
• SelfAttention pooling(channel-dependent in ECAPA_TDNN)
• MultiHeadAttention pooling

Metric

• AmSoftmax
• AAMSoftmax
• AP+Softmax

Released Model Benchmark

NOTE: The test set is VOX1-O(cleaned) dataset and training set is VoxCeleb2-dev. Backend is cosine similarity scoring. The minDCF criterion is the same as VoxSRC2020.

Model	Backbone	Metric	feature	batch size	config	raw EER on Vox1-O(cleaned)	raw DCF	raw EER on Vox1-H	checkpoint
ResNet34-AM-VoxCeleb2	ResNet34	AMSoftmax, scale=30, margin=0.2	81 FBANK(including energy)	128	conf	1.207	0.0738	2.44	ckpt
ResNet34-AM-VoxCeleb2-syncBN	ResNet34	AMSoftmax, scale=30, margin=0.2	81 FBANK(including energy)	128	conf	1.196	0.0791	-	ckpt
SEResNet34-AM-VoxCeleb2	SEResNet34	AMSoftmax, scale=30, margin=0.2	81 FBANK(including energy)	100	conf	1.121	0.0771	2.43	ckpt
SEResNet34-AM-VoxCeleb2-syncBN	SEResNet34	AMSoftmax, scale=30, margin=0.2	81 FBANK(including energy)	100	conf	1.175	0.0745	-	realeased soon
SEResNet34-AM-VoxCeleb2(with checkpointed)	SEResNet34	AMSoftmax, scale=30, margin=0.2	81 FBANK(including energy)	128	conf	1.07	0.0747	2.43	-