claritylab/lucida

Speech and Vision Based Intelligent Personal Assistant

lucida's Issues

Clarify that sirius is beta and insecure

I played around with sirius and detected that the sirius-web app gives you a python terminal if an error occurres. And I think that the whole project is at this time only for developers and should not be served for the public.

Just my minds ;)

sirius unix command shell

I'm working on a sirius client written in C and libcurl.

Usage: sirius [OPTION]... QUESTION ...

  [QUESTION]   QUESTION to ask sirius.
               If it is not specified, data will be
               taken from standard input.

  -a, --speechrecog [WAV FILE]
               WAV_FILE some voice audio file for sirius to recognice.

  -i, --imagematch [IMAGE_FILE]
               IMAGE_FILE some image file for sirius to match.

  -u, --url [URL]
               URL from sirius host..

  -c, --cert [CERTIFICATE_FILE]
               CERTIFICATE_FILE the certificate for sirius host.

  -v, --version
               Display the version number of sirius.

      --verbose
               Display verbose information to stderr.

  -h, --help   Display this help message.

This would allow to pipe questions to sirius:

echo "What is this?" | sirius -i ./eifel_tower.bmp

#Maybe it should answer voice recognition by default if no other flag is set.
sirius -s ./capital_of_italy.wav | sirius 

Error establishing a database connection

When I was going through the Sirius website, I found this error: Error establishing a database connection

[sirius-suite] Add DNN (ASR) brainiac kernel

TODO:
-Update pretrained model
-Make bigger input ?

Innstalation and first run - pocketsphinx empty string

Firstly, thank you very much for so interesting approach!
When i run pocketsphinx_continuous -infile where.is.las.vegas.wav
I got a lot of information plus ... there is las vegas ... so it seems to work.

But! When I run ./start-asr-server.sh pocketsphinx and parallel
Starting server on localhost:8081, use to stop

$ ./sirius-asr-test.sh ../inputs/questions/where.is.las.vegas.wav
Your audio file is:
../inputs/questions/where.is.las.vegas.wav
Sending request to server...

***********************************************

and the server writes transcript []: (empty)
So, the question - why?

I am using last version 8b71aba for now.

Also I can't compile kaldi:

$ sudo ./compile-sirius-servers.sh
[sudo] password for timopheym:
Sphinx4 server done.
Java Pocketsphinx server done.
libtool: install: warning: relinking `libsphinxad.la'
libtool: install: warning: relinking `_sphinxbase.la'
libtool: install: warning: relinking `libpocketsphinx.la'
libtool: install: warning: relinking `_pocketsphinx.la'
C++ Pocketsphinx server done.
***configure failed: Could not find file /include/fst/fst.h:
    you may not have installed OpenFst.  See ../tools/INSTALL *** ```

So, installation process is not simple... ;) (it took me about 2 hours), but I hope we will go through it..

[sirius-suite] fixup dnn-asr

Just making a todo list, I don't want this to sound like its bad!

• stats printed
• Makefile flags

Installation problem as following http://sirius.clarity-lab.org/sirius-suite/#setup

sudo ./compile-sirius-servers.sh raises following erroneous output

Sphinx4 server done.
Java Pocketsphinx server done.
make: *** No targets specified and no makefile found.  Stop.
make: *** No rule to make target `install'.  Stop.
make: *** No targets specified and no makefile found.  Stop.
make: *** No rule to make target `install'.  Stop.
C++ Pocketsphinx server done.
***configure failed: Could not find file /include/fst/fst.h:
    you may not have installed OpenFst.  See ../tools/INSTALL ***

and if I run ./start-qa-server.sh, it raises also this

Error: Could not find or load main class info.ephyra.OpenEphyraServer

OpenCV installation has the following error as well

/home/retina18/Downloads/sirius/sirius-application/opencv-2.4.9/modules/gpu/src/nvidia/core/NCVPixelOperations.hpp(132): error: a storage class is not allowed in an explicit specialization

/home/retina18/Downloads/sirius/sirius-application/opencv-2.4.9/modules/gpu/src/nvidia/core/NCVPixelOperations.hpp(133): error: a storage class is not allowed in an explicit specialization

31 errors detected in the compilation of "/tmp/tmpxft_0000433a_00000000-21_NCVPyramid.compute_35.cpp1.ii".
CMake Error at cuda_compile_generated_NCVPyramid.cu.o.cmake:264 (message):
  Error generating file
  /home/retina18/Downloads/sirius/sirius-application/opencv-2.4.9/build/modules/gpu/CMakeFiles/cuda_compile.dir/src/nvidia/core/./cuda_compile_generated_NCVPyramid.cu.o


make[2]: *** [modules/gpu/CMakeFiles/cuda_compile.dir/src/nvidia/core/./cuda_compile_generated_NCVPyramid.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[1]: *** [modules/gpu/CMakeFiles/opencv_gpu.dir/all] Error 2
make: *** [all] Error 2

I guess something is wrong in installation scripts.

I am on Ubuntu 14.04 with GPU support.

[sirius-suite] testing

Framework to build and test all kernels. Depends on completion of #9.

Completion of error handling

Would you like to add more error handling for return values from functions like the following?

• malloc ⇒ main
• pthread_create ⇒ main

update pocketsphinx models with newly released models

http://cmusphinx.sourceforge.net/2015/01/us-english-acoustic-model-update/

GMM input file missing

The gmm benchmarks in sirius-suite refer to a non-existent input file gmm/input/gmm_data.txt

[sirius-web] Fatal error when trying to run web-sirius.py

Here's the error:

Traceback (most recent call last):
File "./web-sirius.py", line 185, in
h = sys.argv[1]
IndexError: list index out of range

I am running Ubuntu 14.04.

remove hardcoded magic number of threads in build scripts

In some buildscripts, the number of threads for concurrent builds is set to a fixed magical number:

https://github.com/jhauswald/sirius/blob/master/sirius-application/get-opencv.sh
https://github.com/jhauswald/sirius/blob/master/sirius-application/compile-sirius-servers.sh

here it is probably kind of ok, i guess travis does not do concurrent builds:
https://github.com/jhauswald/sirius/blob/master/.travis.yml

I am not sure, that 4 is always for everyone a good choice, as this number is completely system dependent. Usually you would default to non concrrent builds (-j 1) and give the user the option to set the number of threads manually or you could query nproc(1) to find out the number of cores.

[sirius-suite] serial crf

• Make a serial CRF tag.c timing the same component that is parallelized in the pthread version.

broken link?

wget http://kaldi-asr.org/downloads/build/2/sandbox/online/egs/fisher_english/s5/exp/tri5a/graph/archive.tar.gz -O graph.tar.gz
in prepare.sh, has anyone downloaded the file sucessfully?
failed: connection refused. any alternative sources?

where is libfst.so?

Finished building, I try to run but i get

./start-asr-server.sh
Starting server on localhost:8081, use to stop
..//src/online2bin/online2-wav-nnet2-latgen-faster: error while loading shared libraries: libfst.so.1: cannot open shared object file: No such file or directory
^CTraceback (most recent call last):
File "./start-asr-dnn-server.py", line 82, in
server.serve_forever()
File "/usr/lib/python2.7/SocketServer.py", line 236, in serve_forever
poll_interval)
File "/usr/lib/python2.7/SocketServer.py", line 155, in _eintr_retry
return func(*args)
KeyboardInterrupt

Make input only package

For those who start from the repo instead of the package online, it would be useful to have just a package with sirius-suite inputs and another with just the Wikipedia database to not have to download everything again. This covers #21 #28 #29

Build new databases

Collect new databases for:

• OpenEphyra
• Image Matching

[sirius-suite] timing

Design framework to run each kernel X number of times and output statistics:

• run-to-run variability
• average run-time
• average speedup (when applicable)
• input size

Output in easily parse-able format (json, csv).

test wav non existant

In get-kaldi
I could not resolve http://www.singallogic.com/help/EngSamples/Orig/ENG_M.wav

[sirius-suite]: IMM kernels overlap

fix overlap issue when saving keypoints to image.

Broken link

http://www.singallogic.com/help/EngSamples/Orig/ENG_M.wav

prepare.sh want something from this broken link.

Also, I had to go download the big graph model with my browser, wget always times out because of the delay.

[sirius-suite] add ground-truth

Absolute GM:

• regex
• stemmer
• gmm
• crf
• dnn-asr

Approximate GM:

• fe
• fd

readme link leads to server which responds with 500

I get a 500 from http://sirius.clarity-lab.org/sirius-web :( The mail address provided ([email protected]) sends me a mail back with:

The error that the other server returned was:
550 No Such User Here

Do not advise users to run "./compile-sirius-servers.sh" with root privileges

The installation instructions (http://sirius.clarity-lab.org/sirius/) advise users to run various scripts with root privileges:

$ sudo ./compile-sirius-servers.sh

The problem here is, that compilers are NOT designed to be run as root and build scripts must not be designed to require root privileges, when just building the programs or libs.

"./compile-sirius-servers.sh" does not look like it requires root privileges, so I guess updating the installation instructions would be a first step.

Kaldi requires openfst

It doesn't seem like installing openfst is scripted. My first thought is to go download the source:

http://www.openfst.org/twiki/bin/view/FST/FstDownload

We'll see how that works out for me.

libsdl1.2-dev will not install on Ubuntu 14.04

One of the packages installed by get-opencv.sh is libsdl1.2-dev. This fails to install on a fresh Ubuntu 14.04 Desktop. Errors below:

$ sudo apt-get install libsdl1.2-dev
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libcheese-gtk23 : Depends: libclutter-gtk-1.0-0 (>= 0.91.8) but it is not going to be installed
                   Depends: libcogl15 (>= 1.15.8) but it is not going to be installed
 libcheese7 : Depends: libclutter-gst-2.0-0 (>= 0.10.0) but it is not going to be installed
              Depends: gstreamer1.0-clutter but it is not going to be installed
 libclutter-1.0-0 : Depends: libcogl-pango15 (>= 1.15.8) but it is not going to be installed
                    Depends: libcogl15 (>= 1.15.8) but it is not going to be installed
E: Error, pkgProblemResolver::Resolve generated breaks, this may be caused by held packages.

This error causes the entire command to fail, which will prevent all of those dependencies from being installed.

reserved identifier violation

I would like to point out that identifiers like "_MEMORYMAN_H_" and "_PTHREADMAN_H_" do eventually not fit to the expected naming convention of the C language standard.
Would you like to adjust your selection for unique names?

[sirius server] protobuf version error

sirius server doesn't compile with protobuf 2.6.1-1, the dependency on protobuf 2.5.0 is strict?
Can the installer warn of it?

https://gist.github.com/Alessandro-Barbieri/0a47f4739719e8da74ac#file-sirius-protobuf-error

travis-ci for fanciness

Travis-CI is this awesome continuous integration tool that helps us to keep track of the compiling and building process.
More importantly, it will give us a cute icon in our README.md file similar to other open-source projects.

Create a Vagrantfile to automate the process of creating a development environment

Since the process of getting started is quite lengthy, I would suggest that a Vagrantfile be setup with the necessary configuration to make it a one command install (basically vagrant up).

This will provide a replicable development environment for developers that may be interested in contributing to sirius.

[sirius-application] add sphinx4 config to pocketsphinx

Use parameters from S4 on PS, hopefully it increases the accuracy.

Licensing issues

Add Kaldi into end-to-end speech recognition pipeline

Clarify that sirius will currently only build and run on Ubuntu

From reading over the build scripts, it is apparent, that this will only build on Ubuntu. (dependency on apt-get and adding ppas)
In the README fies, this is not as apparent and the installation says:

Sirius has been tested on Ubuntu 12.04 and 14.04.

Which could be more correctly formulated as:

Sirius requires Ubuntu 12.04 or 14.04.

Verbose install script output

Currently, the install scripts, particularly compile-sirius-servers.sh, are redirecting the output of a number of compile commands to /dev/null. This leads to very long stretches of time with no terminal output. We know we're compiling here, why not show command output on screen to give at least some indication of progress? So something like

make -j $THREADS 1>/dev/null

becomes:

make -j $THREADS

QA and Sphinx4 servers hang "insufficient number of threads..."

When running ./start-qa-server.sh, there's an issue with Jetty if the machine has a large number of threads and THREADS in ./start-qa-server.sh is smaller than the total threads, the server hangs and can't accept any requests.

Message: "...insufficient number of threads configured for SelectChanelConnector..."

Similar issue happens with ./start-asr-server.sh sphinx4.

Temporary workaround is to export THREADS=(total # threads available) although none of the services require more than a few (4 max).

libtool not found

I'm trying to build sirius on debian and I found something strange when I build kaldi tools.

After decompress tools.tar.gz, I'm trying to launch 'make' and I had this :

root@sirius:/home/sirius/sirius-application/speech-recognition/kaldi/tools# make
extras/check_dependencies.sh
extras/check_dependencies.sh: libtool is not installed.
extras/check_dependencies.sh: we recommend that you run (our best guess):
sudo apt-get install libtool

APT say it's already installed :
apt-get install libtool
libtool is already the newest version.

I looked the file "extras/check_dependencies.sh" and you use 'which' to check if packages are installed.
In my config "which libtool" doesn't return something so the script stops.

I remove "libtool" of the check list and all running yet (yes it's dirty !)

for f in make automake autoconf patch awk grep bzip2 gzip wget git; do
if ! which $f >&/dev/null; then
echo "$0: $f is not installed."
add_packages $f $f
fi
done

I don't known if it's a bug but I report-it if I can help someone

:)

boost lib dependency should be added in the get_dependencies.sh?

since it's required by building the match in IMM module

HTTP server for C programs

Decided to go with Python wrapper

Merge openephyra-resurrected into sirius

Merge this version of OpenEphyra into Sirius:

https://github.com/TScottJ/OpenEphyra

[sirius-suite] Add pthread functionality to ASR DNN brainiac kernel

[sirius-suite] gpu stemmer

go through this kernel and figure out what is happening.

[sirius-application] figure out how to build a new OE database

Add 'sox' to the get-dependencies.sh script

After installing all components (with no problems) on Ubuntu 14.x, I get the sox command not found when trying out Kaldi. Sox did not get installed. Maybe add sudo apt-get install sox to get-dependencies.sh

opencv sources

Why git clone opencv (you say you need 2.4.9) when Debian have 2.4.9.1 in repository?

Better documentation

Unify the format of all the README.me files once the structure of the repository is settled down.
Write better documentation and copy them into the wiki page.

about performace of kaldi

when I test the ASR part of kaldi, with any example wav file, the server should take more than 10 seconds to response the answer. Is the time cost normal for a PC?

ffmpeg requires PPA on Ubuntu

get-dependencies.sh seems to think Ubuntu provides ffmpeg. They don't. You can work around this using a PPA as described:-

http://www.noobslab.com/2014/12/ffmpeg-returns-to-ubuntu-1410.html

git as superuser

Please don't git clone as superuser.
Thanks.