How to do inference with the pre-trained model

[tuanad121]

How to do inference with the pre-trained model

[tuanad121]

cool, I can it. Here's the script:

import constants as c
from pre_process import extract_features
from models import convolutional_model
from utils import get_last_checkpoint_if_any

from scipy.io.wavfile import read
import numpy as np


def clipped_audio(x, num_frames=c.NUM_FRAMES):
    if x.shape[0] > num_frames + 20:
        bias = np.random.randint(20, x.shape[0] - num_frames)
        clipped_x = x[bias: num_frames + bias]
    elif x.shape[0] > num_frames:
        bias = np.random.randint(0, x.shape[0] - num_frames)
        clipped_x = x[bias: num_frames + bias]
    else:
        clipped_x = x

    return clipped_x


if __name__ == '__main__':
    model = convolutional_model()
    last_checkpoint = get_last_checkpoint_if_any(c.CHECKPOINT_FOLDER)
    print(last_checkpoint)

    model.load_weights(last_checkpoint)
    _, utt1 = read('demo/87-121553-0002.wav')
    utt1 = utt1 / (2**15 - 1)
    feat1 = extract_features(utt1)
    feat1 = clipped_audio(feat1)
    feat1 = feat1[np.newaxis, ...]

    _, utt2 = read('demo/103-1240-0002.wav')
    utt2 = utt2 / (2**15 - 1)
    feat2 = extract_features(utt2)
    feat2 = clipped_audio(feat2)
    feat2 = feat2[np.newaxis, ...]
    print(feat1.shape, feat2.shape)
    emb1 = model.predict(feat1)
    emb2 = model.predict(feat2)

    # similarity
    mul = np.multiply(emb1, emb2)
    s = np.sum(mul, axis=1)
    print(s)

[Walleclipse]

In the inference, you do not need to prepare anchor ...
The general inference process is as follows:

1. Prepare data:
   Get the audio, and extract MFCC features, just similar the training preprocess. Please view details in function extract_features which is located in pre_process.py
2. Predict on model :
   embedding = model.predict(input)
   Then you will get 512-dimensional embedding vector.
3. Use this embedding vector to verification or classification tasks.

Suppose there are two utterances that need to be verified if they were made by the same person.
We first get the embedding vector for 2 utterances
emb1= model.predict(utt1); emb2= model.predict(utt2)
If the similarity between the two embeddings is greater than a certain threshold (e.g. 0.5), we can determine that the two utterances originated from the same person.
utt1 and utt2 has the same speaker if sim(emb1,emb2)>threshold

[tuanad121]

Thanks for your quick response. Much appreciated.
I will try as you suggested. ^^

[cconst04]

I've tried the above code but each time i get a different similarity value. How is this possible?

[Walleclipse]

I've tried the above code but each time i get a different similarity value. How is this possible?

Hi,
I think the main reason is the random clip in def clipped_autio in the code. The audio was random clipping before feeding into the model if the audio is too long.
You can modify the def clipped_autio to a deterministic clip. (clip the deterministic middle frames of the long audio, or clip different parts of the long audio then average the different results).
Or fixed the random seed during running the code, according to reproducible-results-neural-networks-keras.

[LBShinChan]

That's cool! Thank you very much.