JuliusJS

A speech recognition library for the web

JuliusJS is an opinionated port of Julius to JavaScript.
It actively listens to the user to transcribe what they are saying through a callback.

// bootstrap JuliusJS
var julius = new Julius();

julius.onrecognition = function(sentence) {
    console.log(sentence);
};

// say "Hello, world!"
// console logs: `> HELLO WORLD`

Features:

Real-time transcription
Use the provided grammar, or write your own
100% JavaScript implementation
All recognition is done in-browser through a Worker
Familiar event-inspired API
No external server calls

Quickstart

Using Express 4.0

Grab the latest version with bower

bower install juliusjs --save

Include julius.js in your html

<script src="julius.js"></script>

Make the scripts available to the client through your server

var express = require('express'),
    app     = express();

app.use(express.static('path/to/dist'));

In your main script, bootstrap JuliusJS and register an event listener for recognition events

// bootstrap JuliusJS
var julius = new Julius();

// register listener
julius.onrecognition = function(sentence, score) {
    // ...
    console.log(sentence);
};

Your site now has real-time speech recognition baked in!

Configure your own recognition grammar

In order for JuliusJS to use it, your grammar must follow the Julius grammar specification. The site includes a tutorial on writing grammars.
By default, phonemes are defined in voxforge/hmmdefs, though you might find other sites more useful as reference.

Building your own grammar requires the mkdfa.pl script and associated binaries, distributed with Julius.
On Mac OS X
- Use ./bin/mkdfa.pl, included with this repo
On other OS
- Run emscripten.sh to populate bin with the necessary files

Write a yourGrammar.voca file with words to be recognized

The .voca file defines "word candidates" and their pronunciations.

Write a yourGrammar.grammar file with phrases composed of those words

The .grammar file defines "category-level syntax, i.e. allowed connection of words by their category name."

Compile the grammar using ./bin/mkdfa.pl yourGrammar

The .voca and .grammar must be prefixed with the same name
This will generate yourGrammar.dfa and yourGrammar.dict

Give the new .dfa and .dict files to the Julius constructor

// when bootstrapping JuliusJS
var julius = new Julius('path/to/dfa', 'path/to/dict');

Advanced Use

Configuring the engine

The Julius constructor takes three arguments which can be used to tune the engine:

new Julius('path/to/dfa', 'path/to/dict', options)

Both 'path/to/dfa' and 'path/to/dict' must be set to use a custom grammar

'path/to/dfa'

path to a valid .dfa file, generated as described above
if left null, the default grammar will be used

'path/to/dict'

path to a valid .dict file, generated as described above
if left null, the default grammar will be used

options

options.verbose - if true, JuliusJS will log to the console
options.stripSilence - if true, silence phonemes will not be included in callbacks
true by default
options.transfer - if true, captured microphone input will be piped to your speakers
this is mostly useful for debugging
options.*
Julius supports a wide range of options. Most of these are made available here, by specifying the flag name as a key. For example: options.zc = 30 will lower the zero-crossing threshold to 30.
Some of these options will break JuliusJS, so use with caution.
A reference to available options can be found in the JuliusBook.
Currently, the only supported hidden markov model is from voxforge. The h and hlist options are unsupported.

Examples

Voice Command

Coming soon...

Keyword Spotting (e.g., API integration)

Coming soon...

In the wild

If you use JuliusJS let me know, and I'll add your project to this list (or issue a pull request yourself).

Coming soon...

Motivation

Implement speech recognition in...
100% JavaScript - no external dependencies
A familiar and easy-to-use context
- Follow standard eventing patterns (e.g., onrecognition)
As far as accessibility, allow...
Out-of-the-box use
- Minimal barrier to use
  - This means commited sample files (e.g. commited emscripted library)
- Minimal configuration
  - Real-time (opinionated) use only
    - Hide mfcc/wav/rawfile configurations
Useful examples (not so much motivation, as my motivating goals)
- Voice command
- Keyword spotting

Future goals

Better sample recognition grammar (improves out-of-the-box usability)
Examples

Developers

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Build from source

You'll need emscripten, the LLVM to JS compiler, to build this from the C source. Once you have that, run ./emscript.sh. If you are missing other tools, the script will let you know.

As emscript.sh reloads and recompiles static libraries, ./reemscript.sh is available once you've already run emscript.sh. reemscript.sh will only recompile to JavaScript based on your latest changes. This can also be run with npm make.

Additionally, tests ~~are set~~ will be made to run using npm test.
In the meantime, a blank page with the JuliusJS library can be served using npm start.

Codemap

emscript.sh / reemscript.sh

These scripts will compile/recompile Julius C source to JavaScript, as well as copy all other necessary files, to the js folder.

emscript.sh will also compile binaries, which you can use to create recognition grammars or compile grammars to smaller binary files. These are copied to the bin folder.

src

This is where the source for Julius will go once emscript.sh is run. emscript.sh will replace certain files in src/julius4 with those in src/include in order to make src/emscripted, the files eventually compiled to JavaScript.

src/include/julius/app.h - the main application header
src/include/julius/main.c - the main application
src/include/julius/recogloop.c - a wrapper around the recognition loop
src/include/libjulius/src/adin_cut.c - interactions with a microphone
src/include/libjulius/src/m_adin.c - initialization to Web Audio
src/include/libjulius/src/recogmain.c - the main recognition loop
src/include/libsent/configure[.in] - configuration to add Web Audio
src/include/libsent/src/adin/adin_mic_webaudio.c - input on Web Audio

Files in bold were changed to replace a loop with eventing, to simulate multithreading in a Worker.

js

The home to the testing server run with npm start. Files are copied to this folder from dist with emscript.sh and reemscript.sh. If they are modified, they should be commited back to the dist folder.

dist

The home for committed copies of the compiled library, as well as the wrappers that make them work: julius.js and worker.js. dist/listener/converter.js is the file that actually pipes Web Audio to Julius (the compiled C program).

JuliusJS is a port of the "Large Vocabulary Continuous Speech Recognition Engine Julius" to JavaScript

sysrena / juliusjs Goto Github PK

juliusjs's Introduction