khairulhasanmd/esp32_kws

ESP32 Keyword Spotting

YouTube EN | RU

~$ git clone --recursive --depth 1 https://github.com/42io/esp32_kws.git
~$ cd esp32_kws

BUILD ENV

~$ docker build --no-cache -t idf-3.3.4 - < Dockerfile
~$ docker run --rm -it -v $PWD:/home/src --device=/dev/ttyUSB0 idf-3.3.4

KEYWORD SPOTTING

Default models are pre-trained on 0-9 words: zero one two three four five six seven eight nine.

MFCC

Simple non-streaming neural network mode. Model receives the whole mfcc input sequence and then returns the classification result. Jupyter:

~$ make -C mfcc-nn defconfig size erase_flash flash monitor

MFCC STREAMING

Streaming neural network mode. Model receives portion of the input sequence and classifies it incrementally. Jupyter:

~$ make -C mfcc-nn-streaming defconfig size erase_flash flash monitor

TRANSFER LEARNING

Transfer learning is usually done for tasks where your dataset has too little data to train a full-scale model from scratch. TL consists of taking features learned on one problem, and leveraging them on a new, similar problem. For instance, features from a model that has learned to identify english speech may be useful to kick-start a model meant to identify russian.

Embedding | Synthesize | Transfer

~$ make -C mfcc-nn-streaming menuconfig # kws => ru
~$ make -C mfcc-nn-streaming size erase_flash flash monitor

MFCC STREAMING DOUBLE

A false positive error, or false positive, is a result that indicates a given condition exists when it does not. This example consists of two neural networks: first is for classification, second for confirmation. Confirmation allows to reduce false positives. Musan | Libri:

~$ make -C 2-mfcc-nn-streaming defconfig size erase_flash flash monitor