daanzu/deepspeech-websocket-server

Server & client for DeepSpeech using WebSockets for real-time speech recognition in separate environments

deepspeech-websocket-server's Issues

Support for keyword spotting while real time inference

Hi @daanzu Is there a possibility on how we can add keyword spotting while inferencing in real time. Like spotting custom words or giving priority to certain words on the fly just like phrases in Google Speech API. Also, would like to ask whether has Deepspeech come up with real time in the original version? Quick answers appreciated.

Can we make this code suit for deepspeech0.1?

Can we make this code suit for deepspeech0.1?

allowing nested calls after recognition

I would like to chain additional processing steps after the recognition has been completed. This allows the inclusion of other cool things to be executed on top of the speech alone: sentiment analysis, topic understanding, speaker detection, etc.

Here's a rough sketch of the concept...

• Each may follow their own "server" file that launches a new service so that they don't complicate the existing single-server architecture
• Each would communicate over web calls (REST) to avoid process confusion; in the future, we could expand it to be something more rigorous like a message queue.
• Each can communicate via stored JSON/metadata or audio files written to disk
• Each can "register" itself with the main speech server as a secondary process on start-up. For example, the "speaker detection" module will (a) launch it's own service, (b) register with primary server, (c) accept REST calls and reply with JSON / text as required
• Just one example, but each module could leverage other OSS like uis-rnn or pyannote-audio (both taken from this great repo of examples)

Seeking opinions at this point with more details to be flushed out later. Of course, eventually we may convert this suite into a package (e.g. satisfying #2), but that's not paramount right now.

How to speed up performance?

Hi, I would like to know,
How to speed up the performance of websocket?
Is there any parameter that we can tweak?
Does deepspeech and deepspeech-gpu make a difference?
I mean during loading the model at the server.

Thanks for reply.

500 Internal Server Error on localhost?

Hi,

I'm getting the following error when i run the following command on anaconda prompt (Windows 10):

python server.py --model models/output_graph.pbmm --alphabet models/alphabet.txt --lm models/lm.binary --trie models/trie

...and on the localhost, when i navigate to http://127.0.0.1:8080/recognize i get the 500 error:

I have installed all requirements, including pyaudio and the latest deepspeech with pip install deepspeech and pip install deepspeech --upgrade I downloaded the pretrained model from https://github.com/mozilla/DeepSpeech/releases, tried with deepspeech-0.5.1-models.tar.gz, deepspeech-0.5.0-models.tar.gz and deepspeech-0.3.0-models.tar.gz. Still no luck.

Is this a bug? or does someone have a solution for this?

Thanks.

Feature request: ability to easily pause listening

I'll try to have an explore myself but I was wondering if there was a straightforward approach that would allow for the listening to be paused.

Would something with calling stop_stream on the Pyaudio stream and then later starting it hi again work?

Alternatively if there's progress on #2 then I could possibly integrate it that way, having the process start and stop when required.

can't get server running

deepspeech is running fine on my WSL.
Trying to point the server to the right directory but it won't recognize. I tried different paths but still the same. Is there anything special?

python deepspeech-websocket-server/server.py --model deepspeech-0.5.1-models/ -l -t
Initializing model...
2019-10-29 PM 04:44:28.804: main: INFO: (): ARGS.model: deepspeech-0.5.1-models/output_graph.pb
2019-10-29 PM 04:44:28.805: main: INFO: (): ARGS.alphabet: deepspeech-0.5.1-models/alphabet.txt
Traceback (most recent call last):
File "deepspeech-websocket-server/server.py", line 60, in
model = deepspeech.Model(ARGS.model, N_FEATURES, N_CONTEXT, ARGS.alphabet, BEAM_WIDTH)
AttributeError: 'module' object has no attribute 'Model'

move audio writing to server from client

I would like the server to be responsible for logging new audio files to disk instead of the client so that it can intercept incoming requests from both the file API as well as WebSocket. Additionally, this utilizes the EOS marker that clearly indicates that the server should stop recording a stream.

• transplant code using this function into something around this point

This will be a self-completed issue, but seeking opinion before execution.

Concurrent streams cause strange recognitions

Should investigate.

Update to Coqui STT

Hi there @daanzu 👋

this websocket server is an excellent project... thank you for sharing it with the open source community 💚 I used it some back when I was working on DeepSpeech:)

You might not know, but Coqui STT is the new, actively maintained fork of DeepSpeech. It would be a fairly easy upgrade, and the new Coqui v1.0 model is much more accurate than the old DeepSpeech models 😄

For more context: mozilla/DeepSpeech#3693

Give client a Python interface

The sound detection and websocket streaming you've implemented is fantastic. However I'm interested in using the client in another Python application. Do you think we could look into cleaning it up and making it importable? The CLI should stay obviously, but ideally there'd be a single entry point function that could be called by some other Python code, or by the CLI/argparse interface.

I'll write a PR for this if you're interested (and once I get some time).