pankaj-baranwal/pocketsphinx

Updated ROS bindings to pocketsphinx

ROS package for pocketsphinx

Original repository: https://github.com/mikeferguson/pocketsphinx

Also used repo: https://github.com/gorinars/ros_voice_control

You can know more about pocketpshinx here: https://cmusphinx.github.io/

This package is an attempt to bring offline speech recognition to ROS. Pocketsphinx already offers many easy-to-use features in this domain, hence this package can be considered as an extension of pocketsphinx in the ROS world!

Dependencies

1. pyaudio

   sudo pip install pyaudio
   

   If this does not work, follow instructions below:

   sudo apt-get install libasound-dev
   sudo apt-get install python-pyaudio
   

2. pocketsphinx: You will need to have pip preinstalled for this to work

   sudo pip install pocketsphinx
   

   There are many dependencies which need to be met before installation of pocketsphinx through pip works. Use Synaptics package manager to install the unmet dependencies which would be mentioned as error messages on the terminal window in case installation fails. Some of them include:
   libpulse-dev
   swig

Getting Started

Clone this repository into the src folder of your catkin workspace using:

cd ~/catkin_ws/src
git clone https://github.com/Pankaj-Baranwal/pocketsphinx

To know more about catkin workspace and ROS, follow instructions at: http://wiki.ros.org/ROS/Tutorials/InstallingandConfiguringROSEnvironment
After everything is setup, open a terminal from your catkin workspace and type the following command:

catkin_make

Now, you are all setup. Go through the sections in wiki to find out more about how to use this package!

pocketsphinx's Issues

Separate HMM and lm

hmm and lm paths should be separate.
add ~hmm and ~lm option

when kwlist or grammar is used, lm is not needed

hmm is always needed.

Voice Recognition

Can We use pocketPhinx Library for continuous Voice recognition like amazone Alexa.

voice_cmd.launch error

Hello,

Here is my testing env:

ROS kinect version

Ubuntu 16.04.5 TLS xenial

VirtualBox 5.2.18

When type roslaunch pocketsphinx voice_cmd.launch with error:

wilson@wilson-VirtualBox:~$ roslaunch pocketsphinx voice_cmd.launch ... logging to /home/wilson/.ros/log/1dde0bca-34e1-11e9-95df-080027787a29/roslau nch-wilson-VirtualBox-2200.log
Checking log directory for disk usage. This may take awhile.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://wilson-VirtualBox:44835/

SUMMARY

PARAMETERS

• /recognizer/dict: /home/wilson/catk...
• /recognizer/lm: /home/wilson/catk...
• /rosdistro: kinetic
• /rosversion: 1.12.14

NODES
/
recognizer (pocketsphinx/recognizer.py)
voice_cmd_vel (pocketsphinx/voice_cmd_vel.py)

auto-starting new master
process[master]: started with pid [2211]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 1dde0bca-34e1-11e9-95df-080027787a29
process[rosout-1]: started with pid [2224]
started core service [/rosout]
process[recognizer-2]: started with pid [2227]
process[voice_cmd_vel-3]: started with pid [2228]
/usr/lib/python2.7/dist-packages/gtk-2.0/gtk/init.py:57: GtkWarning: could not open display
warnings.warn(str(e), gtk.Warning)
/home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/voice_cmd_vel.py:24: SyntaxWarning: The publisher should be created with an explicit keyword argument 'queue_size'. Please see http://wiki.ros.org/rospy/Overview/Publishers%20and%20Subscribers for more information.
self.pub = rospy.Publisher('cmd_vel', Twist)
/home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py:68: SyntaxWarning: The publisher should be created with an explicit keyword argument 'queue_size'. Please see http://wiki.ros.org/rospy/Overview/Publishers%20and%20Subscribers for more information.
self.pub = rospy.Publisher('~output', String)
Traceback (most recent call last):
File "/home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 177, in
start = recognizer()
File "/home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 73, in init
self.start_recognizer()
File "/home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 80, in start_recognizer
self.pipeline = gst.parse_launch(self.launch_config)
glib.GError: no element "gconfaudiosrc"
Traceback (most recent call last):
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/core.py", line 466, in signal_shutdown
h()
File "/home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 131, in shutdown
gtk.main_quit()
RuntimeError: called outside of a mainloop
[recognizer-2] process has died [pid 2227, exit code 1, cmd /home/wilson/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py __name:=recognizer __log:=/home/wilson/.ros/log/1dde0bca-34e1-11e9-95df-080027787a29/recognizer-2.log].
log file: /home/wilson/.ros/log/1dde0bca-34e1-11e9-95df-080027787a29/recognizer-2*.log

Here is recognizer-2.log

[rospy.client][INFO] 2019-02-20 15:28:34,170: init_node, name[/recognizer], pid[2227]
[xmlrpc][INFO] 2019-02-20 15:28:34,171: XML-RPC server binding to 0.0.0.0:0
[xmlrpc][INFO] 2019-02-20 15:28:34,171: Started XML-RPC server [http://wilson-VirtualBox:45971/]
[rospy.init][INFO] 2019-02-20 15:28:34,172: ROS Slave URI: [http://wilson-VirtualBox:45971/]
[rospy.impl.masterslave][INFO] 2019-02-20 15:28:34,172: _ready: http://wilson-VirtualBox:45971/
[xmlrpc][INFO] 2019-02-20 15:28:34,172: xml rpc node: starting XML-RPC server
[rospy.registration][INFO] 2019-02-20 15:28:34,173: Registering with master node http://localhost:11311
[rospy.init][INFO] 2019-02-20 15:28:34,272: registered with master
[rospy.rosout][INFO] 2019-02-20 15:28:34,273: initializing /rosout core topic
[rospy.rosout][INFO] 2019-02-20 15:28:34,285: connected to core topic /rosout
[rospy.simtime][INFO] 2019-02-20 15:28:34,293: /use_sim_time is not set, will not subscribe to simulated time [/clock] topic
[rosout][INFO] 2019-02-20 15:28:34,316: Launch config: gconfaudiosrc
[rosout][INFO] 2019-02-20 15:28:34,336: Starting recognizer...
[rospy.core][INFO] 2019-02-20 15:28:34,343: signal_shutdown [atexit]
[rospy.impl.masterslave][INFO] 2019-02-20 15:28:34,361: atexit

Thanks

No such file or directory: 'testing_audio.wav'

Running the script and saying each word is working fine, but after the STOP RECORDING message is shown, I get an error at line 236 saying there is no testing_audio.wav file. Earlier, I get the message "sh: rec: command not found", leading me to believe that the file is never created or recorded to. Any insight into what I can do for this? Thanks!

Add feat.params setting

Otherwise accuracy is bad

config.set_string('-featparams', path_to_feat.params"))

glib.GError: no element "vader"

I installed your kinetic-devel branch on a machine running Ubuntu 16.04 and ROS Kinetic. When I run one of the sample launch files (e.g. voice_cmd.launch), recognizer.py crashes with the following error:

Traceback (most recent call last):
  File "/cvgl2/u/pgoebel/jr2_catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 177, in <module>
    start = recognizer()
  File "/cvgl2/u/pgoebel/jr2_catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 73, in __init__
    self.start_recognizer()
  File "/cvgl2/u/pgoebel/jr2_catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 80, in start_recognizer
    self.pipeline = gst.parse_launch(self.launch_config)
glib.GError: no element "vader"

set_kws_threshold.py: Multiple words in keyphrase IndexError

Hi,

I have a problem with your set_kws_threshold.py script. If I use it with the following dictionary and keyword file I get an IndexError.

keyword file:

oh mighty computer /1e-49/
hello world /1e-49/
other phrase /1e-49/
go to one /1e-49/
go to two /1e-49/
go to three /1e-49/
go to four /1e-49/
go to five /1e-49/
start recording /1e-49/
stop recording /1e-49/
cancel recording /1e-49/
open camera /1e-49/
close camera /1e-49/
new action /1e-49/
new location /1e-49/
remember this /1e-49/
next screen /1e-49/
previous screen /1e-49/

dictionary:

oh OW
mighty M AY T IY
computer K AH M P Y UW T ER
hello HH AH L OW
hello(2) HH EH L OW
world W ER L D
other AH DH ER
phrase F R EY Z
go G OW
to T UW
to(2) T IH
to(3) T AH
one W AH N
two T UW
three TH R IY
four F AO R
five F AY V
start S T AA R T
recording R AH K AO R D IH NG
recording(2) R IH K AO R D IH NG
stop S T AA P
cancel K AE N S AH L
open OW P AH N
camera K AE M ER AH
close K L OW S
close(2) K L OW Z
new N UW
new(2) N Y UW
action AE K SH AH N
location L OW K EY SH AH N
remember R IH M EH M B ER
remember(2) R IY M EH M B ER
this DH IH S
next N EH K S T
previous P R IY V IY AH S
screen S K R IY N

IndexError

['oh mighty computer', 'hello world', 'other phrase', 'go to one', 'go to two', 'go to three', 'go to four', 'go to five', 'start recording', 'stop recording', 'cancel recording', 'open camera', 'close camera', 'new action', 'new location', 'remember this', 'next screen', 'previous screen']
Traceback (most recent call last):
  File "set_kws_threshold.py", line 301, in <module>
    preprocess_files(DIC_FILE, KWLIST_FILE)
  File "set_kws_threshold.py", line 54, in preprocess_files
    spaces = _content[indices[0]].count(' ') + spaces + 1
IndexError: list index out of range

Sorry to bother you with such a little problem, but I don't know python and that is why the error causing line exceeds my knowledge. But the line causes a problem, so you should probably know anyway.

BTW: The demo you've provided worked for me, so there should be no problem with my setup.

Thx for the work at cmusphinx 👍
Have a nice day 👋

addyi

No language model specified?

Hi Sir,
I'm trying to run your example kws demo.

I launched:
roslaunch pocketsphinx kws.launch dict:=/home/arifrahman/catkin_ws/src/pocketsphinx/demo/voice_cmd.dic kws:=/home/arifrahman/catkin_ws/src/pocketsphinx/demo/voice_cmd.kwlist

But I got error:
[ERROR] [WallTime: 1512091141.478785] No language model specified. Couldn't find defaut model.

The .dic .kwlist files are there in the directory. Can you give some insights why this happens?

error recognizer.py

while launching robocup.launch i get this error:

process[recognizer-2]: started with pid [6117]
[INFO] [1508098044.496404]: Launch config: gconfaudiosrc
[INFO] [1508098044.504315]: Starting recognizer...
Traceback (most recent call last):
File "/home/hannes/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 177, in
start = recognizer()
File "/home/hannes/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 73, in init
self.start_recognizer()
File "/home/hannes/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 80, in start_recognizer
self.pipeline = gst.parse_launch(self.launch_config)
glib.GError: Kein Element »vader«
Traceback (most recent call last):
File "/opt/ros/kinetic/lib/python2.7/dist-packages/rospy/core.py", line 467, in signal_shutdown
h()
File "/home/hannes/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py", line 131, in shutdown
gtk.main_quit()
RuntimeError: called outside of a mainloop
[recognizer-2] process has died [pid 6117, exit code 1, cmd /home/hannes/catkin_ws/src/pocketsphinx/0.4/nodes/recognizer.py __name:=recognizer __log:=/home/hannes/.ros/log/7508c7a4-b1e4-11e7-a2c9-685d43fb8a01/recognizer-2.log].
log file: /home/hannes/.ros/log/7508c7a4-b1e4-11e7-a2c9-685d43fb8a01/recognizer-2*.log

do have an idea how to fix this error? thx

Pocketsphinx in Ubuntu 20.04 and Python3?

Does this package support python3 or Ubuntu 20.04 systems?

False keyword Detection

Hello,

Pocketsphinx is not behaving good for me. I guess there is a problem with defining thresholds.

I am using this model. It kept on listening to the keywords though i speak nothing.
roslaunch pocketsphinx kws.launch dict:=/home/ros/catkin_ws/src/pocketsphinx/demo/voice_cmd.dic kws:=/home/ros/catkin_ws/src/pocketsphinx/demo/voice_cmd.kwlist hmm:=/home/ros/Desktop/cmusphinx-en

Can you please help with this? So, it must only recognize words when spoken and not randomly.

Please note: I am following everything from your files. SO the keyword file (and everything else) is the same from your medium article.