mideind/fluttersnowboy

Flutter plugin for Snowboy DNN-based hotword detection

Flutter Snowboy plugin

This repository contains the source code to the Flutter Snowboy package. Snowboy is a cross-platform DNN-based hotword detection toolkit implemented in C++. This package makes it easy to use Snowboy in your Flutter app.

Currently only iOS and Android are supported.

Models

The Flutter Snowboy package requires a working Snowboy detection model (pmdl) to be useful. To train your own model, clone this repo and follow the instructions.

How to use

Add dependency to project

Add this to the dependencies list in your pubspec.yaml file:

  flutter_snowboy: ">=0.1.2"

Initialize detector

import 'package:flutter_snowboy/flutter_snowboy.dart';

...

// Instantiate
var detector = Snowboy();

// Load model and other resources.
// This is a moderately expensive operation since it involves file I/O.
var success = detector.prepare("/absolute/path/to/model.pmdl");

Start

void hwHandler() {
    print("Hotword detected");
}

detector.hotwordHandler = hwHandler;

// ... get audio data as UInt8List (e.g. via flutter_sound) and feed into
// the detect function. Audio data should be 16 kHz, 16-bit mono PCM.

detector.detect(data);

// ... and hwHandler() gets called when the hotword is detected in audio data.

Contributing

All contributions are welcome. If you would like to lend and hand, feel free to fork this repository and open pull requests.

Version History

• 0.1.2 - Various fixes (20-05-2023)
• 0.1.1 - Null safety + minor fixes. Now requires Dart 2.12+ (12-01-2022)
• 0.1.0 - Initial release (24-08-2021)

License

flutter_snowboy is Copyright (C) 2021-2023 Miðeind ehf.
Snowboy is Copyright (C) 2016-2020 KITT.AI

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0 or here

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

fluttersnowboy's Issues

Unable to init Snowboy, required bundled file 'common.res' missing

Hi there,

Thanks for making this package, but I haven't get it working till now, always got 'common.res' missing problem. I downloaded the file from https://github.com/Kitt-AI/snowboy/blob/master/resources/common.res and load it in runtime as well.

What I did:

1. install all packages
2. put the pmdl file , common.res , ding.wav in the root's assets folder and declared in pubspec.yml

flutter:
  assets:
     - assets/yoyo.pmdl
     - assets/common.res 
     - assets/ding.wav

The code is learned from the example in https://pub.dev/packages/flutter_snowboy

...

    final directory = await getApplicationDocumentsDirectory();
    final modelPath = "${directory.path}/yoyo.pmdl";
    final commonResPath = "${directory.path}/common.res";
    
    final bytes = await rootBundle.load("assets/yoyo.pmdl");
    final modelData = bytes.buffer.asUint8List();
    await File(modelPath).writeAsBytes(modelData);
    debugPrint('>>> modelPath: $modelPath'); 

    final commonResBytes = await rootBundle.load("assets/common.res");
    final commonResData = commonResBytes.buffer.asUint8List();
    await File(commonResPath).writeAsBytes(commonResData);
    debugPrint('>>> commonResPath: $commonResPath');

    detector = Snowboy();
    await detector.prepare(modelPath); // working, otherwise, it will show the model missing
    detector.hotwordHandler = hotwordHandler;
    await configureAudioSession();
...

The console display:

flutter: >>> modelPath: /var/mobile/Containers/Data/Application/46445C07-C970-468C-B548-D63578DC4BC3/Documents/yoyo.pmdl
flutter: >>> commonResPath: /var/mobile/Containers/Data/Application/46445C07-C970-468C-B548-D63578DC4BC3/Documents/common.res
Unable to init Snowboy, required bundled file 'common.res' missing

I found the similar discussion here: Kitt-AI/snowboy#309
But I don't find a way to pass the common.res path in this lib.