snowboy.service (snowboy.py) crashing

[shivasiddharth]

snowboy.service (snowboy.py) crashing

[shivasiddharth]

What are the modifications that you did ? Also try commenting out the DETECT_DING line. You will anyway get a audio tone that i have created in assistant.py. DING is not required.

[11mail22]

I just reloaded the new files with actions.py and let it run totally unchanged, with same behavior!
Maybe I gave to fast the commands in sequence and also mixed with google questions, and failed questions but it is not stable in comparison with google sdk and same setup.

Here the output, from the first trail fail of the latest snowboy src:
Does this write underflow also occure on your side?

(env) pi@raspberryp GassistPi/snowboy $ python snowboy.py
Listening... Press Ctrl+C to exit
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:33:17
INFO:assistant:Recording audio request.
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:33:40
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:33:50
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:34:04
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:34:38
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
INFO:assistant:Finished playing assistant response.
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:34:52
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:08
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:20
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:33
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
INFO:assistant:Finished playing assistant response.
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:42
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
WARNING:root:SoundDeviceStream write underflow (size: 1600)
INFO:assistant:Finished playing assistant response.
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:35:51
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 06:36:04
Traceback (most recent call last):
File "snowboy.py", line 58, in
sleep_time=0.03)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 172, in start
callback()
File "snowboy.py", line 41, in detected
snowboydecoder.play_audio_file(snowboydecoder.DETECT_DING)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 50, in play_audio_file
rate=ding_wav.getframerate(), input=False, output=True)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 750, in open
stream = Stream(self, *args, **kwargs)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 441, in init
self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable
Traceback (most recent call last):
(env) pi@raspberrypi:~/GassistPi/snowboy $

[shivasiddharth]

it looks like a mic issue. try changing the mic.

[11mail22]

Just changed mic, with same behavior.
btw. I changed also the bing playing, as you proposed, without improvement.
It seems to be a protocol issue. After some working commands, it stucked in google assistant, meaning when saying "Alexa" google answered with "sorry I am Google Assistant" and it stays there until I say "stop". After this, alexa hotword detection is working again but stucks immediately in Google Assitant after the next command - each time. In this mode the crash comes very soon after some less commands.
This behavior I observed multible time.
With command I mean mix commands like "What time is it" and "trigger kitchen lights on".
How it can be ensured that Google listening is closed before entering snowboy, again?

[11mail22]

Now I can easly reproduce the crash: Simple and boring, hope you have an idea...

Saying "Alexa", Bing/Bing, "Stop" repeat this 9 times and it crashes, also with 4sec Pause after each word.

The "Stop" leads to END_OF_UTTERANCE and not to your "stop" implementation.
Tried to add "return continue_conversation" in " if resp.event_type == END_OF_UTTERANCE:" without improvement.

Could there be any multiple instantiation somehow?

Sorry for piecemeal information.

(env) pi@raspberrypi GassistPi/snowboy $ python snowboy.py
Listening... Press Ctrl+C to exit
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:55:52
INFO:assistant:Recording audio request.
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:55:57
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:03
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:10
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:15
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:20
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:25
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:31
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 10:56:38
Traceback (most recent call last):
File "snowboy.py", line 58, in
sleep_time=0.03)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 172, in start
callback()
File "snowboy.py", line 41, in detected
snowboydecoder.play_audio_file(snowboydecoder.DETECT_DING)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 50, in play_audio_file
rate=ding_wav.getframerate(), input=False, output=True)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 750, in open
stream = Stream(self, *args, **kwargs)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 441, in init
self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable
Traceback (most recent call last):
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 80, in audio_callback
Exception return play_data, pyaudio.paContinue
AttributeError: 'NoneType' object has no attribute 'paContinue'
AttributeError in <function _remove at 0x757070b0> ignored

(env) pi@raspberrypi GassistPi/snowboy $ python snowboy.py
Listening... Press Ctrl+C to exit
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:15:49
INFO:assistant:Recording audio request.
WARNING:root:SoundDeviceStream read overflow (3200, 6400)
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:15:57
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:09
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:24
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:33
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:46
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:16:57
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:17:09
INFO:assistant:Recording audio request.
INFO:assistant:End of audio request detected
INFO:snowboy:Keyword 1 detected at time: 2017-10-09 11:17:20
Traceback (most recent call last):
File "snowboy.py", line 58, in
sleep_time=0.03)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 172, in start
callback()
File "snowboy.py", line 41, in detected
snowboydecoder.play_audio_file(snowboydecoder.DETECT_DING)
File "/home/pi/GassistPi/snowboy/snowboydecoder.py", line 50, in play_audio_file
rate=ding_wav.getframerate(), input=False, output=True)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 750, in open
stream = Stream(self, *args, **kwargs)
File "/home/pi/env/local/lib/python2.7/site-packages/pyaudio.py", line 441, in init
self._stream = pa.open(**arguments)
IOError: [Errno -9985] Device unavailable
(env) pi@raspberrypi:~/GassistPi/snowboy $ ~

[shivasiddharth]

Any piece of code can crash. There are so many factors which will cause a crash. That is why I created a service that would restart the snowboy if it crashed. if you run the snowboy as a service, even if it crashes it will restart. If you comment out the snowboy startup audio and run it as a service, the crashes will not be noticeable.
And in the next update, i will try to deactivate snowboy when google assistant starts and activate it back when google assistant stops.

[11mail22]

I analyzed more in detail. The root cause of the crash is in the assistant.py after calling the Actions (and following things) which lead finally to return the function, but google sending in meanwhile the "not understand" answer, which will not be processed.
I could imagine that a pipe got an overflow, after a while, even audio device is restarted by next request.
I avoided this by clearing the audio_out buffer in case of actions and let process everything the same as with each google assist conversation (without pre-returning).

Now it runs stable so far:

My solution look like this (Action returns True if a valid command was found):
if resp.result.spoken_request_text:
usrcmd=resp.result.spoken_request_text
if Action(str(usrcmd).lower()):
self.logger.info('ACTION done')
actionflag = True
self.logger.info('Transcript of user request: "%s".',resp.result.spoken_request_text)
self.logger.info('Playing assistant response.')
if len(resp.audio_out.audio_data) > 0:
#self.logger.info('audio data: %s', resp.audio_out)
if actionflag is False:
self.conversation_stream.write(resp.audio_out.audio_data)
else:
resp.ClearField('audio_out')

Another memory leak still exists. After each command the reserved memory grows about 64-128bytes. When running as python (not service) and stop if by CTRL+C, you will see the number of used commands by voice break and snowboy finished informations, so i seems a stack will be pushed and not fully released. Any idea? New issue?
DEBUG:snowboy:finished.
DEBUG:snowboy:detect voice break
DEBUG:snowboy:finished.
DEBUG:snowboy:detect voice break
DEBUG:snowboy:finished.
DEBUG:snowboy:detect voice break
DEBUG:snowboy:finished.
....

[shivasiddharth]

I have an idea to modify the Ok-google main.py script to be triggered using snowboy. But i am too busy to work on it and moreover i just updated the project. So when i find time, i will update that..
Thanks for your suggestion regarding changes to assistant.py..
Regarding the memory usage i think it is due to the logs.