mtg/gaia

C++ library to apply similarity measures and classifications on the results of audio analysis, including Python bindings. Together with Essentia it can be used to compute high-level descriptions of music.

Gaia 2

ABOUT

Gaia is a C++ library with python bindings which implements similarity measures and classifications on the results of audio analysis, and generates classification models that Essentia can use to compute high-level description of music.

Licence: Affero GPLv3 license

Documentation: http://essentia.upf.edu/documentation/gaia

Dependencies:

• Qt >= 4.5
• libYAML >= 0.1.1
• Python >= 2.4
• SWIG >= 1.3.31
• Eigen >= 3.3.4

INSTALL

Linux

• Install dependencies (Ubuntu/Debian)

  apt-get install build-essential libqt4-dev libyaml-dev swig python-dev pkg-config libeigen3-dev
  

  Note that Gaia build will fail if you are using swig 3.0.8. Install either a previous or later version. You will encounter this problem if you are using swig package distributed with Ubuntu 16.04. In this case install the newest swig version from source (https://github.com/swig/swig).

• Online help for WAF (build system)

  ./waf --help
  

• Configure with the desired options:

  ./waf configure --download [--with-python-bindings] [--with-stlfacade] [--with-asserts]
  

  NOTE: in order to link Essentia library with Gaia, do not use the --with-stlfacade option

• Compile libgaia.a:

  ./waf
  

• Install (to install system-wide you might need sudo)

  ./waf install [--destdir=/where/ever/]
  

• Build documentation (optional), it will be located at build/doc/ folder

  python src/doc/regenerate_docstring.py
  

MacOS

Build from command-line using Homebrew (recommended):

• Install Qt4:

  brew install cartr/qt4/qt
  

• Install homebrew tap:

  brew tap MTG/essentia
  

• Build Gaia (this will also install all the dependencies)

  brew install gaia --HEAD
  

Build from command-line:

• Install python, qt libraries 4.8, libYAML and swig dependencies using Homebrew:

  brew install python
  
  brew install swig libyaml cartr/qt4/qt@4
  

• Install pyyaml pip package in case you want to build with python bindings:

  pip install pyyaml
  

• Configure and build similarly to Linux (see above).

Build with QtCreator (alternative):

Gaia2lib

• Install qt libraries 4.8 (including debug libraries) and QtCreator from the Qt download archives.

• Install libYAML and swig dependencies using Homebrew (we assume you already have a python installation, otherwise you can also install it using Homebrew):

  brew install swig libyaml
  

• Use QtCreator to open the project file 'Gaia2lib.pro' in packaging/darwin/Gaia2lib/

• Compile the project (you will probably need to configure QtCreator to work with your Qt installation and also to set up the build folder for the project)

Gaia2Python - python bindings

• Use swig to generate the file 'gaia_wrap.cxx' that will be needed to compile 'Gaia2Python':

  swig -c++ -python -w451 /path_to_gaia_source/src/bindings/gaia.swig
  

• Copy the generated 'gaia_wrap.cxx' to the Gaia2Python project folder:

  cp /path_to_gaia_source/src/bindings/gaia_wrap.cxx /path_to_gaia_source/packaging/darwin/Gaia2Python/
  

• Use QtCreator to open the project file 'Gaia2Python.pro' in packaging/darwin/Gaia2Python/ and compile

• Run ./make_release_tarball in packaging/darwin:

  ./make_release_tarball
  

• Copy the folder packaging/darwin/tmp/gaia2/python/gaia2 (created when running make_release_tarball.sh) to the site-packages directory of your python distribution. You can now import gaia2 from python.

Windows

• Use the QtCreator projects inside the packaging/win32 directory.

3RD PARTY

This library contains source code from the LibSVM project, which is distributed under the revised BSD license. Please refer to the src/3rdparty/libsvm/COPYRIGHT file for more information.

This library contains the Mersenne Twister random number generator, which is distributed under the BSD license.

This library contains source code from the Alglib project, which is distributed under the 3-clause BSD license.

This library contains source code from FrogLogic command line parser which is distributed under the BSD license.

gaia's Issues

Brew install gaia does not work on macOS Sierra

I upgraded my mac to Sierra and I can't install gaia via homebrew. I wrote "brew install gaia" and I received the message below. Do you have any recommendations?

"qt: This formula either does not compile or function as expected on macOS
versions newer than El Capitan due to an upstream incompatibility.
Error: An unsatisfied requirement failed this build.
"

Installation script bug

waf configure will sometimes fail due to _get_python_variables in (what looks like a generated file) .../wafadmin/Tools/python.py not supporting the strings returned from the script it executes being wrapped in " instead of '.

def _get_python_variables(python_exe,variables,imports=['import sys']):
...
elif s[0]=="'"and s[-1]=="'":
return_values.append(s[1:-1])
#Possible Fix:
elif s[0]=='"' and s[-1]=='"':
return_values.append(s[1:-1])
.....

I can't use the library svm.h and svm.o with gaia

Hey, I must to use the svm.h in my code, but I can't use it. When I try to link the svm.h and svo.o with my code it is returned an error. The error says that svm.o and svm.cpp was fisrt difined in libgaia2.a
I found the the svm.h and svm.o here -> /home/jefferson/gaia-master/src/3rdparty/libsvm

The big problem is because I want to use the functions of svm and I can not use it. What do I need to put in my makefile to link svm.h and svm.o with my code that uses essentia and gaia???
I REALLY need the answer.

Not able to use gaia2 in python

While configuring gaia2 in ubuntu15.04 using the ./waf configure [--with-python-bindings], although it says built successfully, but also prints the line "No function [--with-python-bindings] defined in /home/kartik/open_source/acoustic_server/gaia/wscript" in the end and I am not able to install gaia2 in python, it simply gives an import error.

could not load the waf tool swig from ................

when i execute ./waf configure --with-python-bindings, it shows couldn't load the waf tool swig

when i execute ./waf configure --download --with-python-bindings. it shows could not load the Waf tool

Don't know how to fix this one, is it a problem about the version? wish some boss could help me

Build failed

Build failed
Traceback (most recent call last):
File "/home/echo66/tese/features-extraction/gaia-master/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Task.py", line 123, in process
ret=self.run()
File "/home/echo66/tese/features-extraction/gaia-master/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Task.py", line 47, in run
return m1(self)
File "/home/echo66/tese/features-extraction/gaia-master/src/wscript", line 53, in write_pcfile
open('../build/gaia2.pc', 'w').write(bld.env.pcfile)
IOError: [Errno 2] No such file or directory: '../build/gaia2.pc'

Add support for one-class SVMs

Add support for one-class SVMs to Gaia as the LibSVM already supports them. This classifier will be useful in the context of AcousticBrainz.

What's the problem of "Could not open file: /Videos/gender_pro_dir/datasets/gender.db_0_20.partdb"

Thanks for giving me the instruction to generate a ".history" file .
But I have a problem when following the instruction to run train_model.py script , that is "Exception: Could not open file: /home/gender_project_dir/datasets/gender.db_0_20.partdb".
so, how to solve this problem?

Examples?

Hi,

are there any examples available of using the gaia library within a C framework?
I've been looking for an "examples" folder, but the only examples I could find are python examples.
For instance it would be nice to have the moodtagger in C form.

Maybe I'm just overlooking something...
Thanks again,
Daniele

Build files with swig 3.0.3 (osx yosemite)

Hey,
I realised building gaia fails on osx yosemite when using swig 3.0.3.
I had to rollback swig to 3.0.2 to fix the issue.

Build failed on osx yosemite "Error: Unknown SWIG preprocessor directive"

I am trying to install on yosemite, but keep failing to build. I tried ./waf clean and configuring again but i get the following errors:

[ 85/104] swig: src/bindings/gaia.swig -> build/src/bindings/gaia2.swigwrap_1.cxx build/src/bindings/gaia2.py
../src/bindings/frozendataset.i:57: Error: Unknown SWIG preprocessor directive: try (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:14: Error: Unknown SWIG preprocessor directive: shortcut (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:27: Error: Unknown SWIG preprocessor directive: until (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:42: Error: Unknown SWIG preprocessor directive: converts (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:52: Error: Unknown SWIG preprocessor directive: useful (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:56: Error: Unknown SWIG preprocessor directive: oh (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:57: Error: Unknown SWIG preprocessor directive: but (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:58: Error: Unknown SWIG preprocessor directive: FilterWrapper (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:61: Error: Unknown SWIG preprocessor directive: make (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:71: Error: Unknown SWIG preprocessor directive: make (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:92: Error: Unknown SWIG preprocessor directive: complete (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:127: Error: Unknown SWIG preprocessor directive: args (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:190: Error: Unknown SWIG preprocessor directive: print (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:203: Error: Unknown SWIG preprocessor directive: remove (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:220: Error: Unknown SWIG preprocessor directive: automatic (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:223: Error: Unknown SWIG preprocessor directive: real (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:231: Error: Unknown SWIG preprocessor directive: commutativity (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:235: Error: Unknown SWIG preprocessor directive: string (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:241: Error: Unknown SWIG preprocessor directive: enhanced (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:243: Error: Unknown SWIG preprocessor directive: should (if this is a block of target language code, delimit it with %{ and %})
../src/bindings/pythonic.i:244: Error: Unknown SWIG preprocessor directive: reach (if this is a block of target language code, delimit it with %{ and %})
Waf: Leaving directory `/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/build'

Build failed
-> task in '_gaia2' failed (exit status 1):
{task 4345091344: swig gaia.swig -> gaia2.swigwrap_1.cxx,gaia2.py}
['/usr/local/bin/swig', '-c++', '-python', '-w451', '-o', '/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/build/src/bindings/gaia2.swigwrap_1.cxx', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/build/src', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/src', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/build/src/bindings', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/src/bindings', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/build/src/metrics', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/src/metrics', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/build/src/3rdparty', '-I/Users/erikparr/Documents/_Projects/freesound-python-master/gaia-master/src/3rdparty', '-I/usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/include/python2.7', '-I/usr/local/Cellar/qt/4.8.7/include', '-I/usr/local/Cellar/qt/4.8.7/include/QtCore', '-I/usr/local/Cellar/qt/4.8.7/include/QtNetwork', '-DGAIA_VERSION="2.4.4"', '-DGAIA_GIT_SHA="Undefined"', '-DHAVE_QTUITOOLS_DEBUG=1', '-DHAVE_QTCORE=1', '-DHAVE_QTGUI=1', '-DHAVE_QTUITOOLS=1', '-DHAVE_QTNETWORK=1', '-DHAVE_QTOPENGL=1', '-DHAVE_QTSQL=1', '-DHAVE_QTSVG=1', '-DHAVE_QTTEST=1', '-DHAVE_QTXML=1', '-DHAVE_QTXMLPATTERNS=1', '-DHAVE_QTWEBKIT=1', '-DHAVE_QTHELP=1', '-DHAVE_QTSCRIPT=1', '-DHAVE_QTDECLARATIVE=1', '-DPYTHONDIR="/usr/local/lib/python2.7/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python2.7/site-packages"', '-DHAVE_PYTHON_H=1', '-DNDEBUG', '-DQT_SHARED', '-DQT_SHARED', '../src/bindings/gaia.swig']

Add support for loading json files

Add support for loading json files into Gaia. Only sig (yaml) files are supported currently.

Build failed on Debian

Error message:
no module named swig

But I had swig installed. Is waf tool 'swig' a different entity?

building with python bindings fails, probably b/c of python 3

Configure completes with no errors:
python2 ./waf configure --download --with-python-bindings --with-tests

Building via
python2 ./waf
fails with error below. running ./waf clean and reconfiguring without python bindings worked with no errors.
`

[ 85/101] swig: src/bindings/gaia.swig -> build/src/bindings/gaia2.swigwrap_1.cxx build/src/bindings/gaia2.py
swig: parser.y:667: update_defaultargs: Assertion `!DohGetattr(n, "defaultargs")' failed.
Waf: Leaving directory `/home/rich/git/gaia/build'
Build failed
 -> task in '_gaia2' failed (exit status -6): 
    {task 140524425009680: swig gaia.swig -> gaia2.swigwrap_1.cxx,gaia2.py}
['/usr/bin/swig', '-c++', '-python', '-w451', '-o', '/home/rich/git/gaia/build/src/bindings/gaia2.swigwrap_1.cxx', '-I/home/rich/git/gaia/build/src', '-I/home/rich/git/gaia/src', '-I/home/rich/git/gaia/build/src/bindings', '-I/home/rich/git/gaia/src/bindings', '-I/home/rich/git/gaia/build/src/metrics', '-I/home/rich/git/gaia/src/metrics', '-I/home/rich/git/gaia/build/src/3rdparty', '-I/home/rich/git/gaia/src/3rdparty', '-I/usr/include/python3.5m', '-I/usr/include/qt4', '-I/usr/include/qt4/QtCore', '-DGAIA_VERSION="2.4.4"', '-DGAIA_GIT_SHA="v2.4.4-22-gf418151"', '-DHAVE_QTCORE=1', '-DHAVE_QTGUI=1', '-DHAVE_QTUITOOLS=1', '-DHAVE_QTNETWORK=1', '-DHAVE_QTOPENGL=1', '-DHAVE_QTSQL=1', '-DHAVE_QTSVG=1', '-DHAVE_QTTEST=1', '-DHAVE_QTXML=1', '-DHAVE_QTXMLPATTERNS=1', '-DHAVE_QTWEBKIT=1', '-DHAVE_QT3SUPPORT=1', '-DHAVE_QTHELP=1', '-DHAVE_QTSCRIPT=1', '-DHAVE_QTDECLARATIVE=1', '-DPYTHONDIR="/usr/local/lib/python3.5/site-packages"', '-DPYTHONARCHDIR="/usr/local/lib/python3.5/site-packages"', '-DHAVE_PYEMBED=1', '-DHAVE_PYEXT=1', '-DHAVE_PYTHON_H=1', '-DNDEBUG', '-DNDEBUG', '-DQT_SHARED', '../src/bindings/gaia.swig']

Libs not found on MacOS X (Yosemite)

Hi,

Per the instructions, I installed packages via homebrew and am getting these results:

Setting top to                           : /Users/...
Setting out to                           : /Users/...
→ configuring the project in /Users/...
→ Building in release mode
Checking for program pkg-config          : /usr/local/bin/pkg-config 
Checking for 'yaml-0.1'                  : yes 
Checking for 'g++' (c++ compiler)        : clang++ 
Checking for 'gcc' (c compiler)          : clang 
Checking for program qmake-qt4           : not found 
Checking for program qmake4              : not found 
Checking for program qmake               : /usr/local/bin/qmake 
Checking for program uic-qt3             : not found 
Checking for program uic3                : not found 
Checking for program uic-qt4             : not found 
Checking for program uic                 : /usr/local/bin/uic 
Checking for uic version                 : version 4.8.6 
Checking for program moc-qt4             : not found 
Checking for program moc                 : /usr/local/bin/moc 
Checking for program rcc                 : /usr/local/bin/rcc 
Checking for program lrelease-qt4        : not found 
Checking for program lrelease            : /usr/local/bin/lrelease 
Checking for program lupdate-qt4         : not found 
Checking for program lupdate             : /usr/local/bin/lupdate 
Found the Qt4 libraries in               : /usr/local/Cellar/qt/4.8.6/lib 
Checking for pkg-config version >= '0.1' : yes 
Checking for 'QtCore_debug'              : not found 
Checking for 'QtGui_debug'               : not found 
Checking for 'QtUiTools_debug'           : yes 
Checking for 'QtNetwork_debug'           : not found 
Checking for 'QtOpenGL_debug'            : not found 
Checking for 'QtSql_debug'               : not found 
Checking for 'QtSvg_debug'               : not found 
Checking for 'QtTest_debug'              : not found 
Checking for 'QtXml_debug'               : not found 
Checking for 'QtXmlPatterns_debug'       : not found 
Checking for 'QtWebKit_debug'            : not found 
Checking for 'Qt3Support_debug'          : not found 
Checking for 'QtHelp_debug'              : not found 
Checking for 'QtScript_debug'            : not found 
Checking for 'QtDeclarative_debug'       : not found 
Checking for 'QtCore'                    : yes 
Checking for 'QtGui'                     : yes 
Checking for 'QtUiTools'                 : yes 
Checking for 'QtNetwork'                 : yes 
Checking for 'QtOpenGL'                  : yes 
Checking for 'QtSql'                     : yes 
Checking for 'QtSvg'                     : yes 
Checking for 'QtTest'                    : yes 
Checking for 'QtXml'                     : yes 
Checking for 'QtXmlPatterns'             : yes 
Checking for 'QtWebKit'                  : yes 
Checking for 'Qt3Support'                : not found 
Checking for 'QtHelp'                    : yes 
Checking for 'QtScript'                  : yes 
Checking for 'QtDeclarative'             : yes 
'configure' finished successfully (0.985s)

Of course, when I try to build, it fails. Is there another package I can install with the other QT libs? Any help is appreciated.

–Levi

Specify model id when training classifier models and store it in history file

Add a new option to specify a model_id when training classifiers and store it in the history file.

Where to get the original dataset for trainning the SVM models?

Hello,
There are a little difference between accuracy when using the model from http://essentia.upf.edu/documentation/svm_models/ to classify music my own.
So , please tell me where to get the original dataset for trainning the SVM models so that i can make some
contrast.
Thanks!

Error in ./waf install on ubuntu 14.04

Hi,

Before executing ./waf install , I needed to copy /home/francois/gaia/build/src in /home/francois/gaia/src else I would get an error:

francois@francois-ThinkCentre-M58 ~/gaia $ sudo ./waf install
Waf: Entering directory `/home/francois/gaia/build'
→ building from /home/francois/gaia
Could not find 'libgaia2.a' in /home/francois/gaia/src

Looks like ./waf install is not configured with the right directory to look for the compiled files

Thanks

François Pelletier

Filtering queries with multidimensional descriptor values is not possible

When filtering the results of a query (view.nnSearch(query,metric,filter)) it is only possible to use single dimensional descriptors (ex: WHERE value.lowlevel.pitch.mean > 123). There is no way to filter at a specific dimension of a multidimensional descriptor (ex: WHERE value.lowlevel.mfcc.mean[4] > 12.3).

HELP, Command "['/usr/local/bin/swig'] -version" returned 127

hi, thanks for your work!
git clone the newest source, and swig is 3.0.2.
while execute the configure command
"./waf configure --with-python-bindings" in OS X shell,
got "Command "['/usr/local/bin/swig'] -version" returned 127",
any idea will be appreciated

How to generate a ".history" file?

Hello,
Thanks for give me the tool that to use a ".history" file.
Now I want to generate a model myself,please tell me how to generate a ".history" file?
Thanks.

Add weight parameter for training C-SVC SVMs on unbalanced data

Libsvm supports working with unbalanced data via adding class weights. Add a new parameter for SVM configuration which will allow to configure weights for each class. Currently the weights are unassigned by default, and there is not way to configure them.

Make sure that cross-validation is not affected by unbalanced data as well.

Implement LaSVM classifier (wishlist)

In the case we are limited by the computational complexity of training SVMs on large dataset, we can implement more scalable LaSVM. The source code for LaSVM is released under GPLv2.

Point class in Python bindings doesn't support non-string keys

See first two exceptions in this example:

Python 2.7.6 (default, Jun 22 2015, 17:58:13)                                                                                             
[GCC 4.8.2] on linux2                                                                                                                     
Type "help", "copyright", "credits" or "license" for more information.                                                                    
>>> import gaia2                                                                                                                          
>>> p = gaia2.Point()                                                                                                                     
>>> p[unicode("hello")] = "test"                                                                                                          
Traceback (most recent call last):                                                                                                        
  File "<stdin>", line 1, in <module>                                                                                                     
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 4677, in autoSetValue                                             
    p.setLabel(name, value)                                                                                                               
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 3546, in setLabel                                                 
    return _gaia2.Point_setLabel(self, *args)                                                                                             
NotImplementedError: Wrong number or type of arguments for overloaded function 'Point_setLabel'.                                          
  Possible C/C++ prototypes are:                                                                                                          
    gaia2::Point::setLabel(QString const &,gaia2::StringDescriptor const &)                                                               
    gaia2::Point::setLabel(int,QString const &,gaia2::StringDescriptor const &)                                                           
    gaia2::Point::setLabel(std::string const &,gaia2::StringDescriptor const &)                                                           
    gaia2::Point::setLabel(int,std::string const &,gaia2::StringDescriptor const &)                                                       
    gaia2::Point::setLabel(std::string const &,std::vector< std::string,std::allocator< std::string > > const &)                          
    gaia2::Point::setLabel(int,std::string const &,std::vector< std::string,std::allocator< std::string > > const &)                      
    gaia2::Point::setLabel(std::string const &,std::string const &)                                                                       
    gaia2::Point::setLabel(int,std::string const &,std::string const &)                                                                   

>>> p[0] = "test"                                                                                                                         
Traceback (most recent call last):                                                                                                        
  File "<stdin>", line 1, in <module>                                                                                                     
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 4677, in autoSetValue                                             
    p.setLabel(name, value)                                                                                                               
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 3546, in setLabel                                                 
    return _gaia2.Point_setLabel(self, *args)                                                                                             
NotImplementedError: Wrong number or type of arguments for overloaded function 'Point_setLabel'.                                          
  Possible C/C++ prototypes are:                                                                                                          
    gaia2::Point::setLabel(QString const &,gaia2::StringDescriptor const &)                                                               
    gaia2::Point::setLabel(int,QString const &,gaia2::StringDescriptor const &)                                                           
    gaia2::Point::setLabel(std::string const &,gaia2::StringDescriptor const &)                                                           
    gaia2::Point::setLabel(int,std::string const &,gaia2::StringDescriptor const &)                                                       
    gaia2::Point::setLabel(std::string const &,std::vector< std::string,std::allocator< std::string > > const &)                          
    gaia2::Point::setLabel(int,std::string const &,std::vector< std::string,std::allocator< std::string > > const &)                      
    gaia2::Point::setLabel(std::string const &,std::string const &)                                                                       
    gaia2::Point::setLabel(int,std::string const &,std::string const &)                                                                   

>>> p["test"] = "test"                                                                                                                    
Traceback (most recent call last):                                                                                                        
  File "<stdin>", line 1, in <module>                                                                                                     
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 4677, in autoSetValue                                             
    p.setLabel(name, value)                                                                                                               
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 3546, in setLabel                                                 
    return _gaia2.Point_setLabel(self, *args)                                                                                             
Exception: Couldn't find node with name 'test'                                                                                            
>>>

Get rid of Qt

The reasons behind its use in Gaia were:

• QString that manages Unicode. There's hardly any alternatives to do that in C++ with a simple API.
• QMap and QVector are more easy to use that std C++, although the difference is small.
• Versioned QDataStreams made it easy to load and save datasets with different versions.
• Some more reasons we forgot...

Re-factor Gaia to get rid of dependency on Qt. Take care of Unicode management, there might be surprises. If everything is in UTF-8, std::string should work fine.

Model building scripts don't complete if npermutations < ncpus

train_model.py has some issue where it hangs between testing and selecting the top model.

This was running on a machine with 12 cores, but we modified the profile file so that there was only one permutation. It appears to have run the evaluation correctly, and ordered the "results" by accuracy, but then it failed to execute selectBestModel.
Running select_best_model from the commandline worked fine.

Error when training model - Division by zero when all values in an array are 0

https://acousticbrainz.org/datasets/7a052ef4-cad2-4362-a075-5933b20609f0

Traceback (most recent call last):
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/classification/classificationtask.py", line 195, in <module>
    task.run(className, outfilename, param, dsname, gtname, evalconfig)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/classification/classificationtask.py", line 177, in run
    confusion = evaluateNfold(evalparam['nfold'], ds, gt, trainerFun, **trainingparam)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/classification/evaluation.py", line 116, in evaluateNfold
    classifier = trainingFunc(trainds, traingt, *args, **kwargs)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/classification/classifier_SVM.py", line 35, in train_SVM
    ds = transform(ds, 'normalize', { 'independent': True })
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/__init__.py", line 4601, in transform
    return analyzer.analyze(dataset).applyToDataSet(dataset)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/__init__.py", line 1997, in analyze
    return _gaia2.Analyzer_analyze(self, *args)
Exception: Normalize: Division by zero in .tonal.chords_histogram

Training with own datasets

I've been trying to figure out how i could use gaia with my own data to produce a classifier and i think i've come up against a road block.

When running generate_classification_project.py it bails out looking for 'stable/genre_tzanetakis/metadata/config.yaml'. I guess this is the file that configures the svm to produce the tzanetakis history file. An older version of which i've been able to make use of with essentia to classify my own tracks.

I can't find this configuration file in the repo, is it intentionally missing? If so, would it be possible to get an example of some kind i could start from to figure out how to get going?

train_model.py is hanging somewhere and wait infinitely

Hello
when running train_model.py on ~1200 songs it's hanging when evaluating the results file and waits infinitely.

rename model name in history file

Sometimes we might want to rename the model stored in a history file. See if we can do this without having to re-compute the history file, and instead read and save it

Build fail on OSX due to Swig update

Building Gaia with python bindings fails using the most recent swig 3.0.8 on OSX. Swig 3.0.4 is reported to work fine.

/usr/local/bin/swig -c++ -python -w451 -o /Users/dbogdanov/mtg/gaia/build/src/bindings/gaia2.swigwrap_1.cxx -I/Users/dbogdanov/mtg/gaia/build/src -I/Users/dbogdanov/mtg/gaia/src -I/Users/dbogdanov/mtg/gaia/build/src/bindings -I/Users/dbogdanov/mtg/gaia/src/bindings -I/Users/dbogdanov/mtg/gaia/build/src/metrics -I/Users/dbogdanov/mtg/gaia/src/metrics -I/Users/dbogdanov/mtg/gaia/build/src/3rdparty -I/Users/dbogdanov/mtg/gaia/src/3rdparty -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -I/usr/local/Cellar/qt/4.8.7_2/include -I/usr/local/Cellar/qt/4.8.7_2/include/QtCore -DGAIA_VERSION="2.4.4" -DGAIA_GIT_SHA="v2.4.4-17-gff46634" -DHAVE_QTUITOOLS_DEBUG=1 -DHAVE_QTCORE=1 -DHAVE_QTGUI=1 -DHAVE_QTUITOOLS=1 -DHAVE_QTNETWORK=1 -DHAVE_QTOPENGL=1 -DHAVE_QTSQL=1 -DHAVE_QTSVG=1 -DHAVE_QTTEST=1 -DHAVE_QTXML=1 -DHAVE_QTXMLPATTERNS=1 -DHAVE_QTWEBKIT=1 -DHAVE_QTHELP=1 -DHAVE_QTSCRIPT=1 -DHAVE_QTDECLARATIVE=1 -DPYTHONDIR="/usr/local/lib/python2.7/site-packages" -DPYTHONARCHDIR="/usr/local/lib/python2.7/site-packages" -DHAVE_PYTHON_H=1 -DNDEBUG -DQT_SHARED ../src/bindings/gaia.swig
Assertion failed: (!Getattr(n, "defaultargs")), function update_defaultargs, file parser.y, line 667.
Abort trap: 6

Report test/train splits for n-fold cross-validation when training models

For better reproducibility, the evaluateNfold method should report point ids for test/train splits for each fold (as a json dictionary).

Report support vectors of SVM models (update libsvm)

We are currently using an old verion 2.8 and we can update to the newest version 3.21. There does not seem to be any important updates that would change models behavior according to the changelog. However, one very useful feature was added:

• add sv_indices in model structure, so users can know which training instances are SVs

We should be able to provide such information about support vectors back to Gaia model training scripts and log it.

Not configuring with python binding

It is not getting configured with the --with-python-bindings. What should be done ?
This is the output from the ./waf configure

abc@abc:~/code/gaia$ ./waf configure --download [--with-python-bindings] [--with-asserts] [--with-cyclops]
Setting top to : /home/neo01124/code/gaia
Setting out to : /home/neo01124/code/gaia/build
→ configuring the project in /home/neo01124/code/gaia
→ Building in release mode
Checking for program pkg-config : /usr/bin/pkg-config
Checking for 'yaml-0.1' : yes
Checking for 'g++' (c++ compiler) : /usr/bin/g++
Checking for 'gcc' (c compiler) : /usr/bin/gcc
Checking for program qmake-qt4 : /usr/bin/qmake-qt4
Checking for program qmake4 : not found
Checking for program qmake : /usr/bin/qmake
Checking for program uic-qt3 : not found
Checking for program uic3 : /usr/bin/uic3
Checking for program uic-qt4 : /usr/bin/uic-qt4
Checking for uic version : version 4.8.1
Checking for program moc-qt4 : /usr/bin/moc-qt4
Checking for program rcc : /usr/bin/rcc
Checking for program lrelease-qt4 : /usr/bin/lrelease-qt4
Checking for program lupdate-qt4 : /usr/bin/lupdate-qt4
Found the Qt4 libraries in : /usr/lib/x86_64-linux-gnu
Checking for pkg-config version >= '0.1' : yes
Checking for 'QtCore_debug' : not found
Checking for 'QtGui_debug' : not found
Checking for 'QtUiTools_debug' : not found
Checking for 'QtNetwork_debug' : not found
Checking for 'QtOpenGL_debug' : not found
Checking for 'QtSql_debug' : not found
Checking for 'QtSvg_debug' : not found
Checking for 'QtTest_debug' : not found
Checking for 'QtXml_debug' : not found
Checking for 'QtXmlPatterns_debug' : not found
Checking for 'QtWebKit_debug' : not found
Checking for 'Qt3Support_debug' : not found
Checking for 'QtHelp_debug' : not found
Checking for 'QtScript_debug' : not found
Checking for 'QtDeclarative_debug' : not found
Checking for 'QtCore' : yes
Checking for 'QtGui' : yes
Checking for 'QtUiTools' : yes
Checking for 'QtNetwork' : yes
Checking for 'QtOpenGL' : yes
Checking for 'QtSql' : yes
Checking for 'QtSvg' : yes
Checking for 'QtTest' : yes
Checking for 'QtXml' : yes
Checking for 'QtXmlPatterns' : yes
Checking for 'QtWebKit' : yes
Checking for 'Qt3Support' : yes
Checking for 'QtHelp' : yes
Checking for 'QtScript' : yes
Checking for 'QtDeclarative' : yes
'configure' finished successfully (0.282s)
No function [--with-python-bindings] defined in /home/neo01124/code/gaia/wscript

When running ./waf configure: SyntaxError: invalid syntax except Exception, e:

I am taking a wild guess that this might have something to do with me having python3 as default, only because that has caused trouble other times. Also, are the various config options explained anywhere? If they are in the ./waf --help command, that fails with the same error.

Here is the error:

[gaia]$./waf configure --download 
Waf: The wscript in '/home/rich/gaia' is unreadable
Traceback (most recent call last):
  File "/home/rich/gaia/.waf3-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Scripting.py", line 87, in waf_entry_point
    set_main_module(Context.run_dir+os.sep+Context.WSCRIPT_FILE)
  File "/home/rich/gaia/.waf3-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Scripting.py", line 112, in set_main_module
    Context.g_module=Context.load_module(file_path)
  File "/home/rich/gaia/.waf3-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Context.py", line 281, in load_module
    exec(compile(code,path,'exec'),module.__dict__)
  File "/home/rich/gaia/wscript", line 15
    except Exception, e:
                    ^
SyntaxError: invalid syntax

Various problems running the tests (files not found)

Hello, after downloading and installing the gaia library I wanted to run the tests. Unfortunately, I couldn't get them all to pass. I am getting errors like:

    testValues (test_merge.TestMerge) ... Could not read from file: "data/dataset_small/Techno and Electro/01 Bigmouth.mp3.sig"
    Could not read from file: "data/dataset_small/Blues/01 Boogie Woogie Stomp.mp3.sig"
    Yaml::loadFromFile: Could not open file: data/dataset_small/Techno and Electro/01 Bigmouth.mp3.sig
    Yaml::loadFromFile: Could not open file: data/dataset_small/Blues/01 Boogie Woogie Stomp.mp3.sig

I found many of those files in the folder data/yamaha_small, so I created a symlink named dataset_small pointing to yamaha_small. This doesn't completely solve the problem, but allows almost all tests to pass. Repeatedly running the tests improves the results, but one tests still fails.

Here's a full log of the tests (after creating the symlink):
https://gist.github.com/bxdpxndx/9737934

Thank you very much!

Configuration failed on static builds

Hello,

I guess maybe gaia has something wrong. The message shows as below:
root@lance-virtual-machine:~/Desktop/ess# ./waf configure --with-static-examples
Setting top to : /mnt/hgfs/ess
Setting out to : /mnt/hgfs/ess/build
→ configuring the project in /mnt/hgfs/ess
→ Building in release mode
→ Compiling with static examples on Linux: search for pre-built dependencies in 'packaging/debian'
Checking for 'g++' (c++ compiler) : /usr/bin/g++
Checking for 'gcc' (c compiler) : /usr/bin/gcc
Checking for program pkg-config : /usr/bin/pkg-config
Checking for 'libavcodec' >= 53.25.0 : yes
Checking for 'libavformat' : yes
Checking for 'libavutil' : yes
Checking for 'libswresample' : not found
Checking for 'fftw3f' : yes
Checking for 'samplerate' : not found
The configuration failed
(complete log in /mnt/hgfs/ess/build/config.log)
Does anybody meet the same problem? Thanks.

How to use the ".history" file?

Hello,
What's the usage of the ".history" files in the SVM model ? Please give me some instructions of that.
Thanks.

gaia with Qt5

Hello,
I'm building an application in which I would like to use essentia ( so gaia required ) but this app uses Qt5.
I'm encountering a problem because gaia depends on Qt4, I read this post #7 saying that you're or will be working on an Qt free version. Waiting for that I tried to build gaia with Qt5 but got tons of errors with Qt globals not being defined or something. I got messages "Expected ';' at end of declaration list" for every Qt function declaration ending with a Q_DECL_* macro.
Does anybody have guidelines or successfully built gaia with Qt5 ?
Thanks

Report standard deviation for raw and normalized accuracies when training classifier models

Update training scripts for classifier models:

• Report normalized accuracy with respect to the number of instances per class in addition to raw accuracy.
• Report standard deviation for both raw and normalized accuracy values.

Error when run gaiafusion.py

When I try to run the script "gaiafusion.py" in Gaia ,there is a Exception : Could not open file : /home/music/gender_0_160.partdb .
In the code, the gender_0_160.partdb is just a template file . But it doesn't exists in the directory.
Please give me some instruction on how to run this script .
Thanks !

Infinite loop in train_model_from_sigs.py

The program displays "zero division in tonal.chord_progression".

Python bindings raise `Exception` for some (all?) of its errors

For example,

  File "/usr/local/lib/python2.7/dist-packages/gaia2/classification/classificationtaskmanager.py", line 157, in applyPreprocessing
    ds = transform(ds, step['transfo'], step.get('params', {}))
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 4601, in transform
    return analyzer.analyze(dataset).applyToDataSet(dataset)
  File "/usr/local/lib/python2.7/dist-packages/gaia2/__init__.py", line 1997, in analyze
    return _gaia2.Analyzer_analyze(self, *args)
Exception: PointLayout: pattern "metadata.*" didn't match any descriptor

and many places in the pygaia bindings raise a plain Exception. At the very least this should be a GaiaException, but even better should be something explicit to the task which is being performed.

Install instructions for Mac OSX and linux may be a bit confusing

if you want to install gaia systemwide,
shouldn't the install instructions line:
$ ./waf install [--destdir=/where/ever/]
actually be:
$ sudo ./waf install

this worked for me after we had some hair pulling getting it working on mac OSX mavericks.

Report mean and stdev for accuracies of trained classifiers

Provide stdev for accuracies of trained classifiers in addition to mean across folds when reporting n-fold cross-validation results. Currently, accuracy is computed from the overall confusion matrix which is a sum of confusion matrices for each fold.

The evaluateNfold should gather all confusion matrices apart in a list and return that list.

Error when training model

2016-07-28 10:22:27,631 - INFO - Processing job cba7d880-04ca-4e02-8517-deb5619f3b0f...
INFO     ClassificationTaskManager  |  Merging original base dataset...
2016-07-28 10:22:27,710 - INFO - Merging original base dataset...
Harmonizing chunks so that they all have the same layout & history...
Traceback (most recent call last):
  File "evaluate.py", line 181, in <module>
    main()
  File "evaluate.py", line 35, in main
    evaluate_dataset(pending_job)
  File "evaluate.py", line 69, in evaluate_dataset
    filelist_file=filelist_path,
  File "/home/acousticbrainz/acousticbrainz-server/dataset_eval/../dataset_eval/gaia_wrapper.py", line 47, in train_model
    run_tests(project_file)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/scripts/classification/run_tests.py", line 55, in runTests
    test.run()
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/classification/classificationtaskmanager.py", line 177, in run
    self.mergeDataSet()
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/classification/classificationtaskmanager.py", line 170, in mergeDataSet
    mergeAll(self.conf['filelist'], self.conf['datasetFilename'], 5000, transfoFile = None)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/fusion.py", line 160, in mergeAll
    vldescs, nandescs, rdescs = harmonizeChunks(partfiles)
  File "/home/acousticbrainz/ve/local/lib/python2.7/site-packages/gaia2/fusion.py", line 86, in harmonizeChunks
    ds.load(partfiles[0])
IndexError: list index out of range

Build failure when linking Gaia with Essentia

Got this error when building Essentia with option --with-gaia because include/gaia2 (which contains point.h) seems to be not added to the include path.
Maybe #include "point.h" in parser/parsertypes.h should be replaced with #include "../point.h"? (this resolves the issue for me)

...
In file included from /usr/include/gaia2/parser/filter.h:24:0,
                 from /usr/include/gaia2/searchspace.h:26,
                 from /usr/include/gaia2/searchspacepool.h:24,
                 from /usr/include/gaia2/view.h:27,
                 from /usr/include/gaia2/convert.h:27,
                 from ../src/algorithms/highlevel/gaiatransform.cpp:23:
/usr/include/gaia2/parser/parsertypes.h:24:19: fatal error: point.h: No such file or directory
compilation terminated.
Waf: Leaving directory `/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build'
Build failed
 -> task in 'essentia' failed (exit status 1): 
    {task 140440742811536: cxx gaiatransform.cpp -> gaiatransform.cpp.1.o}
['/usr/bin/g++', '-pipe', '-Wall', '-msse', '-msse2', '-mfpmath=sse', '-O2', '-fPIC', '-pthread', '-D_FORTIFY_SOURCE=2', '-march=x86-64', '-mtune=generic', '-O2', '-pipe', '-fstack-protector-strong', '-w', '-fPIC', '-fPIC', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/essentia', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/essentia', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/essentia/scheduler', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/essentia/scheduler', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/essentia/streaming', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/essentia/streaming', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/essentia/streaming/algorithms', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/essentia/streaming/algorithms', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/essentia/utils', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/essentia/utils', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/3rdparty', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/3rdparty', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/3rdparty/spline', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/3rdparty/spline', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/build/src/3rdparty/vamp-plugin-sdk-2.4', '-I/home/shared/pkgbuilds/essentia-archlinux/src/essentia/src/3rdparty/vamp-plugin-sdk-2.4', '-I/usr/include/taglib', '-I/usr/include/qt4', '-I/usr/include/qt4/QtCore', '-DHAVE_AVCODEC=1', '-DHAVE_AVFORMAT=1', '-DHAVE_AVUTIL=1', '-DHAVE_AVRESAMPLE=1', '-DHAVE_SAMPLERATE=1', '-DHAVE_TAGLIB=1', '-DHAVE_YAML=1', '-DHAVE_FFTW=1', '-DHAVE_GAIA2=1', '-D__STDC_CONSTANT_MACROS', '-DPYTHONDIR="/usr/lib/python2.7/site-packages"', '-DPYTHONARCHDIR="/usr/lib/python2.7/site-packages"', '-DHAVE_PYTHON_H=1', '../src/algorithms/highlevel/gaiatransform.cpp', '-c', '-o', 'src/algorithms/highlevel/gaiatransform.cpp.1.o']
==> ERROR: A failure occurred in build().
    Aborting...

Where is the "external process gaiafusion" in the gaiaLib?

Is there any method to use gaia without Qt?

It's not easy to install Qt without admin authority for me.

Error : 'ConfigurationContext' object has no attribute 'check_tool'

having freshly cloned the waf_refactor_osx branch on mavericks and having installed python (2.7.6), swig (3.0.0), libyaml (0.1.6) and qt (4.8.6) from brew, running this:

./waf configure --with-python-bindings

leads to this:

Setting top to                           : /path/to/gaia 
Setting out to                           : /path/to/gaia/build 
→ configuring the project in path/to/gaia
Checking for program pkg-config          : /usr/local/bin/pkg-config 
Checking for 'QtCore'                    : yes 
Checking for 'yaml-0.1'                  : yes 
Checking for 'g++' (c++ compiler)        : clang++ 
Checking for 'gcc' (c compiler)          : clang 
Traceback (most recent call last):
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Scripting.py", line 97, in waf_entry_point
    run_commands()
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Scripting.py", line 153, in run_commands
    ctx=run_command(cmd_name)
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Scripting.py", line 146, in run_command
    ctx.execute()
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Configure.py", line 128, in execute
    super(ConfigurationContext,self).execute()
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Context.py", line 87, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Context.py", line 128, in recurse
    user_function(self)
  File "/path/to/gaia/wscript", line 181, in configure
    conf.recurse('src')
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Context.py", line 128, in recurse
    user_function(self)
  File "/path/to/gaia/src/wscript", line 23, in configure
    conf.recurse('bindings')
  File "/path/to/gaia/.waf-1.7.9-16e1644c17ba46b94844133bac6e2a8c/waflib/Context.py", line 128, in recurse
    user_function(self)
  File "/path/to/gaia/src/bindings/wscript", line 21, in configure
    conf.check_tool('swig python')
AttributeError: 'ConfigurationContext' object has no attribute 'check_tool'