jmschrei/pomegranate

Fast, flexible and easy to use probabilistic modelling in Python.

pomegranate's Issues

Memory consumption while using train() in parallel with n_jobs > 1

I've been running a dataset containing 30 sequences with 2000 observations each through train/fit with baum-welch. I see the memory consumption creeping up with iterations - this increase becomes really significant when I set n_jobs to be larger than 1 (I am using an 8 core processor): with n_jobs = 6, the memory consumption hits the RAM limit really soon. Is this supposed to happen, or am I doing something wrong in assigning n_jobs?

Thanks!

HMM Training example from docs does not work

I'm seeing this error:

In [1]: from pomegranate import HiddenMarkovModel as HMM

In [2]: hmm = HMM()

In [3]: hmm.train( [[5, 2, 3, 4], [5, 7, 2, 3, 5]], distribution_inertia=0.3, edge_inertia=0.25 )
Exception ignored in: 'pomegranate.hmm.HiddenMarkovModel.log_probability'
TypeError: object of type 'NoneType' has no len()
Exception ignored in: 'pomegranate.hmm.HiddenMarkovModel.log_probability'
TypeError: object of type 'NoneType' has no len()
Exception ignored in: 'pomegranate.hmm.HiddenMarkovModel._baum_welch_update'
TypeError: object of type 'NoneType' has no len()
Exception ignored in: 'pomegranate.hmm.HiddenMarkovModel.log_probability'
TypeError: object of type 'NoneType' has no len()
Exception ignored in: 'pomegranate.hmm.HiddenMarkovModel.log_probability'
TypeError: object of type 'NoneType' has no len()
Training improvement: 0.0
Total Training Improvement: 0.0
Out[3]: 0.0

I'm running this on linux with numpy 1.9.0, scikit-learn 0.17.1, and python 3.4.1

Figure out how to read/write everything easily

jesus christ Jacob it's been a year, get off your ass and figure this out.

Learning algorithms ?

@jmschrei, excellent work with the API - its the most intuitive amongst the several pgm libraries I have used. Are there plans of implementing learning algorithms in pomegranate ?

Exception AttributeError in training a HMM (code was working with yahmm) (referencing #82)

This is a continuation of issue #82

I forgot to mention this the last time - I had tested the code with the already implemented DiscreteDistribution (for a different dataset) and the same error had popped up. I tried the PoissonDistribution today, and the error came up again - I am pasting the code here again (I have commented out my definition of the PoissonDistribution, and am using the version you implemented last time).

And here's the error:
...
Exception AttributeError: 'exit' in 'pomegranate.hmm.HiddenMarkovModel._train_loop' ignored
Total Training Improvement: 0.0
...

The code:
test_pomegranate.txt

Add new values to a bayesian network.

Hello,

I am starting to learn bayesian networks, and this seems a great project to work with. This is my code:

activity = DiscreteDistribution({
    'breakfast': 1./6,
    'lunch': 1./6,
    'dinner': 1./6,
    'hangout': 1./6,
    'call': 1./6,
    'meeting': 1./6
})

duration = ConditionalProbabilityTable([
    ['breakfast', 30, 1.0],
    ['breakfast', 60, 0.0],
    ['lunch', 30, 0.5],
    ['lunch', 60, 0.5],
    ['dinner', 30, 0.0],
    ['dinner', 60, 1.0],
    ['hangout', 30, 0.0],
    ['hangout', 60, 1.0],
], [activity])

# Make the states
s0 = State( activity, name="type")
s1 = State( duration, name="duration")

# Make the bayes net, add the states, and the conditional dependencies.
network = BayesianNetwork( "example" )
network.add_states( [s0, s1] )
network.add_transition( s0, s1 )
network.bake()

My question is how can I add a new activity to the Discrete Distribution, and to the ConditionalProbabilityTable?

Thanks.

hmm-rainy-sunny.ipynb cannot load into Jupyter (unreadable notebook)

Loading the file examples/hmm_rainy_sunny.ipynb into Jupyter 4.0.6 on Python 3.5.1 | Anaconda 2.4.1 (64-bit) gives the error:

Unreadable Notebook: /home/nickyn/github/pomegranate/examples/hmm_rainy_sunny.ipynb
NotJSONError('Notebook does not appear to be JSON: \'{\\n "cells": [\\n {\\n "cell_type": "m...',)

pip install fails on Windows

"pip install pomegranate" on Windows, with Anaconda and Visual C++ for Python (9.0) installed, fails as follows:

BayesianNetwork.c
c:\anaconda\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(12) : Warning Msg: Using deprecated NumPy API, disable it by #defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
pomegranate\BayesianNetwork.c(1842) : error C2433: '__pyx_f_11pomegranate_5utils__log' : 'inline' not permitted on data declarations

Have distributions take in other distributions as parameters

Instead of taking in constants, support the ability to take in distributions as parameters, such as:

NormalDistribution( NormalDistribution( 4, 2 ), LognormalDistribution( 5, 2 ) )

This will be the core of the new Bayesian stuff added in.

o request

serialization with to_json and from_json

(sorry for the long post...)

In the current system, theto_json methods cause the stringification of the inner-most objects (the distributions) with json.dumps and these strings are put in dictionaries by their parents (their containing classes) which in turn are stringified by calls to json.dumps and used by their parents (e.g. for distributions within states within models) . The resulting json has strings within strings so that json.loads(model.to_json()) returns something ugly (i.e. a dictionary where the inner objects are all strings).
I made some code changes so that when the parent object performs to_json, it embeds serialized inner objects with json.loads so there are no more strings within strings.

e.g. the State.to_json() becomes:

def to_json( self ):
    """
    Convert this state to JSON format.
    """

    return json.dumps( {    'class' : 'State',
                'distribution' : None if self.is_silent() else json.loads(self.distribution.to_json()),
                #'distribution' : None if self.is_silent() else str( self.distribution ),
                'name' : self.name,
                'weight' : self.weight
                })

Where similar appropriate changes are done to the from_json code.
For example, the State.from_json code is:

    def from_json( cls, s ):
        """
        Read a State from a given string formatted in JSON.
        """
        
        # Load a dictionary from a JSON formatted string
        d = json.loads( s )

        # If we're not decoding a state, we're decoding the wrong thing
        if d['class'] != 'State':
            raise IOError( "State object attempting to decode {} object".format( d['class'] ) )

        # If this is a silent state, don't decode the distribution
        if d['distribution'] is None:
            return cls( None, str(d['name']), d['weight'] )

        # Otherwise it has a distribution, so decode that
        return cls( Distribution.from_json( json.dumps(d['distribution']) ),
                    name=str(d['name']), weight=d['weight'] )

The result is that now models can be read into simple python structures

For example:

from pomegranate import *
import json
model = HiddenMarkovModel(name="ExampleModel")
distribution = UniformDistribution(0.0, 1.0)
state = State(distribution, name="uniform")
state2 = State(NormalDistribution(0, 2), name="normal")
silent = State(None, name="silent")
model.add_state(state)
model.add_state(state2)

model.add_transition(state, state, 0.4)
model.add_transition(state, state2, 0.4)
model.add_transition(state2, state2, 0.4)
model.add_transition(state2, state, 0.4)

model.add_transition(model.start, state, 0.5)
model.add_transition(model.start, state2, 0.5)
model.add_transition(state, model.end, 0.2)
model.add_transition(state2, model.end, 0.2)

model.bake()
json.loads(model.to_json())

results in:

{u'class': u'HiddenMarkovModel',
 u'distribution ties': [],
 u'edges': [[3, 0, 0.5, 0.5, None],
  [3, 1, 0.5, 0.5, None],
  [0, 0, 0.4, 0.4, None],
  [0, 2, 0.20000000000000004, 0.2, None],
  [0, 1, 0.4, 0.4, None],
  [1, 0, 0.4, 0.4, None],
  [1, 2, 0.20000000000000004, 0.2, None],
  [1, 1, 0.4, 0.4, None]],
 u'end': {u'class': u'State',
  u'distribution': None,
  u'name': u'ExampleModel-end',
  u'weight': 1.0},
 u'end_index': 2,
 u'name': u'ExampleModel',
 u'silent_index': 2,
 u'start': {u'class': u'State',
  u'distribution': None,
  u'name': u'ExampleModel-start',
  u'weight': 1.0},
 u'start_index': 3,
 u'states': [{u'class': u'State',
   u'distribution': {u'class': u'Distribution',
    u'frozen': False,
    u'name': u'NormalDistribution',
    u'parameters': [0, 2]},
   u'name': u'normal',
   u'weight': 1.0},
  {u'class': u'State',
   u'distribution': {u'class': u'Distribution',
    u'frozen': False,
    u'name': u'UniformDistribution',
    u'parameters': [0.0, 1.0]},
   u'name': u'uniform',
   u'weight': 1.0},
  {u'class': u'State',
   u'distribution': None,
   u'name': u'ExampleModel-end',
   u'weight': 1.0},
  {u'class': u'State',
   u'distribution': None,
   u'name': u'ExampleModel-start',
   u'weight': 1.0}]}

So far I modified hmm.pyx, base.pyx and distributions.pyx but I can adapt gmm.pyx and fsm.pyx if you are interested (I haven't used them so far in my own code). Note that nosetests runs smoothly (so I guess it doesn't check serialization)

Let me know if you'd like to make these changes to your package and we can figure out how to do it (a pull request?)

gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32'

Hey guys,

i have trouble installing pomegranate. First i got the same problem as in this issue:
#43

After installing gcc 4.8.4 with homebrew (i'm running OSX) and using it for the compiling process i got the following error:

(ENV)Davids-MacBook-Pro:~ davidmoeller$ pip install pomegranate
Collecting pomegranate
Using cached pomegranate-0.3.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): cython>=0.22.1 in ./ENV/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.8.0 in ./ENV/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): joblib>=0.9.0b4 in ./ENV/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): networkx>=1.8.1 in ./ENV/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): decorator>=3.4.0 in ./ENV/lib/python2.7/site-packages (from networkx>=1.8.1->pomegranate)
Building wheels for collected packages: pomegranate
Running setup.py bdist_wheel for pomegranate
Complete output from command /Users/davidmoeller/ENV/bin/python -c "import setuptools;__file__='/private/var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/pip-build-yMxOa5/pomegranate/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/tmpS3hn_hpip-wheel-:
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-10.11-intel-2.7
creating build/lib.macosx-10.11-intel-2.7/pomegranate
copying pomegranate/__init__.py -> build/lib.macosx-10.11-intel-2.7/pomegranate
running build_ext
building 'pomegranate.BayesianNetwork' extension
creating build/temp.macosx-10.11-intel-2.7
creating build/temp.macosx-10.11-intel-2.7/pomegranate
/usr/local/Cellar/gcc48/4.8.4/bin/gcc-4.8 -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/Users/davidmoeller/ENV/lib/python2.7/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pomegranate/BayesianNetwork.c -o build/temp.macosx-10.11-intel-2.7/pomegranate/BayesianNetwork.o
gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32'
error: command '/usr/local/Cellar/gcc48/4.8.4/bin/gcc-4.8' failed with exit status 1

----------------------------------------
Failed building wheel for pomegranate
Failed to build pomegranate
Installing collected packages: pomegranate
Running setup.py install for pomegranate
Complete output from command /Users/davidmoeller/ENV/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/pip-build-yMxOa5/pomegranate/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/pip-ubIz2L-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/davidmoeller/ENV/include/site/python2.7/pomegranate:
running install
running build
running build_py
running build_ext
building 'pomegranate.BayesianNetwork' extension
/usr/local/Cellar/gcc48/4.8.4/bin/gcc-4.8 -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/Users/davidmoeller/ENV/lib/python2.7/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pomegranate/BayesianNetwork.c -o build/temp.macosx-10.11-intel-2.7/pomegranate/BayesianNetwork.o
gcc-4.8: error: unrecognized command line option '-Wshorten-64-to-32'
error: command '/usr/local/Cellar/gcc48/4.8.4/bin/gcc-4.8' failed with exit status 1

----------------------------------------
Command "/Users/davidmoeller/ENV/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/pip-build-yMxOa5/pomegranate/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/pip-ubIz2L-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/davidmoeller/ENV/include/site/python2.7/pomegranate" failed with error code 1 in /private/var/folders/6h/12sm_tdx4t98pz0gh2h8yc_80000gn/T/pip-build-yMxOa5/pomegranate

Maybe the problem depends on the failed wheel problem? Did someone had an similar error already?

best regards,
David

nosetests fails with a segmentation fault

Hi
I cloned the current version (0.2.8) from github and ran nosetests -w tests/
The tests end in:

................ESegmentation fault (core dumped)

My setup is (as python reports it):

Python 2.7.9 (default, May 10 2015, 09:08:45) 
[GCC 4.8.2] on linux2

>>> numpy.__version__
'1.10.0.dev0+01c59d6'
>>> cython.__version__
'0.22'
>>> scipy.__version__
'0.15.1'
>>> networkx.__version__
'1.9.1'
>>> matplotlib.__version__
'1.4.3'

More understandable errors

I get error like this:

File "pomegranate/BayesianNetwork.pyx", line 72, in pomegranate.BayesianNetwork.BayesianNetwork.bake (pomegranate/BayesianNetwork.c:2681)
KeyError: {
    "weight" : 1.0,
    "name" : "GO:0005488",
    "distribution" : "{\n    \"parameters\" : [\n        {\n            \"false\" : 0.25,\n            \"true\" : 0.75\n        }\n    ],\n    \"frozen\" : false,\n    \"name\" : \"DiscreteDistribution\",\n    \"class\" : \"Distribution\"\n}",
    "class" : "State"
}

It's not very understandable and gives no guidance to fix the problem in my network. I think it's poor error reporting.

HiddenMarkovModel.freeze_distribution() fails

    import pomegranate as pm
    model=pm.HiddenMarkovModel()
    model.add_state(pm.State(pm.NormalDistribution(0,1),'s1'))
    model.freeze_distributions()

In [5]: model.freeze_distributions()
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-29-49df663003f8> in <module>()
----> 1 model.freeze_distributions()

pomegranate/hmm.pyx in pomegranate.hmm.HiddenMarkovModel.freeze_distributions (pomegranate/hmm.c:5076)()

TypeError: 'NoneType' object is not iterable

The same happens if I bake() the model first. I guess it reached the start or end state and tries to freeze its None distribution

Proposal: Possibility to give the HMM(names) a prefix/postfix after initialization

Scenario:

In my project i need to concatenate (sometimes) the same model once, twice or multiple times. To do so, it is necessary to have disjoint nodes. But this is only possible with a renamed model. The error suggests that the network library has methods to add a prefix/postfix.

networkx.exception.NetworkXError: ('The node sets of G and H are not disjoint.', 'Use appropriate rename=(Gprefix,Hprefix)or use disjoint_union(G,H).')

Exception OverflowError: (34, 'Numerical result out of range') in 'pomegranate.hmm.HiddenMarkovModel._baum_welch_update' ignored

When i use a version prior the mentioned one, everything works fine while getting some ZeroDivisionErrors through the training.

After updating to the newest version, the output looks like follows:

You can see the difference with the first Training for a "Jump". The funny part is, that every training with the broken version seems to be different. It is like having a random factor messing something up.

In my system i use Feature Vectors with 25 features and i use a IndependentComponentDistribution with 25 NormalDistributions. (Maybe the problem lies here?) The states of a HMM are aligned at the average length of a Feature Vector chain of the training dataset.

Maybe use SSE version of logsumexp

At least in the HMM, you are spending a lot of time computing the log-sum-exp, I wonder if it would be useful to use a SSE version of that function https://github.com/rmcgibbo/logsumexp

Finding the number of iterations that train() runs till convergence? Also, issue with the output printed at every iteration

2 questions:

1. I am running the train function (with baum-welch) - I am wondering if there's a way I can get the number of iterations that the train loop runs till convergence and the log-likelihood of the model at every iteration step from model instance? Something like a list containing the log likelihood at every iteration step - the length of this list gives the number of iterations. I want to run several seeds and fix the maximum number of iterations to a manageable number and pick the seeds that have hit the stop threshold before the iterations reach that maximum.
2. Every iteration of the train loop prints an output of this kind:
   ...
   hmm: -1071.56869799 -9.30763231015 -1080.8751549
   hmm: -1071.69600328 -9.18032344342 -1080.8751549
   hmm: -1071.82331108 -9.05301459842 -1080.8751549
   hmm: -1071.95061963 -8.92570582563 -1080.8751549
   hmm: -1072.07792841 -8.79839729261 -1080.8751549
   hmm: -1072.20523726 -8.67108955602 -1080.8751549
   hmm: -1072.33254612 -8.54378446468 -1080.8751549
   hmm: -1072.459855 -8.41648815939 -1080.8751549
   ...
   This is probably coming from (hmm.pyx):
   with gil:
   print( "hmm: ", f[(i+1)_m + k], b[(i+1)_m + k], log_sequence_probability)

Is there some way to suppress this from printing? This makes it difficult to see the training improvement at every iteration with verbose=True

Thanks so much!

Sampling from Probability Tables

This is easy for the joint case, but more difficult in the conditional case, because you either need to supply the values for the parents, or the parent distribution itself.

Add functions/classes to readthedocs with autoclass

It would be helpful to be able to easily read through the various classes and their methods on the readthedocs website, similar to http://hmmlearn.readthedocs.org/en/latest/api.html. It seems this would be straightforward, and would require adding a file similar to https://raw.githubusercontent.com/hmmlearn/hmmlearn/master/doc/api.rst. Not entirely sure if that's all that's required, though - I'm not too familiar with readthedocs setup.

HMM Vectorized Updates

Currently the calculation of the emission and transition matrices are done iteratively. It is possible to do this as a vectorized update, and I believe the distributions will natively handle being given a vector. I will look into speeding this update step up, which should benefit Baum-Welch training for all HMMs.

pip install pomegranate error on mac

I am having a bad time installing this module, which is a shame (the packages looks just like something that I might need!)

$ pip install pomegranate
Collecting pomegranate
  Using cached pomegranate-0.3.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): cython>=0.22.1 in ./env/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.8.0 in ./env/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): joblib>=0.9.0b4 in ./env/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): networkx>=1.8.1 in ./env/lib/python2.7/site-packages (from pomegranate)
Requirement already satisfied (use --upgrade to upgrade): decorator>=3.4.0 in ./env/lib/python2.7/site-packages (from networkx>=1.8.1->pomegranate)
Installing collected packages: pomegranate
  Running setup.py install for pomegranate
    Complete output from command /Users/code/Development/notebooks/env/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/mn/nf34qzvd11v3kd023kzyvzm00000gn/T/pip-build-O_xFKk/pomegranate/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/mn/nf34qzvd11v3kd023kzyvzm00000gn/T/pip-SHAxRC-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/code/Development/notebooks/env/include/site/python2.7/pomegranate:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.10-intel-2.7
    creating build/lib.macosx-10.10-intel-2.7/pomegranate
    copying pomegranate/__init__.py -> build/lib.macosx-10.10-intel-2.7/pomegranate
    running build_ext
    building 'pomegranate.BayesianNetwork' extension
    creating build/temp.macosx-10.10-intel-2.7
    creating build/temp.macosx-10.10-intel-2.7/pomegranate
    cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/Users/code/Development/notebooks/env/lib/python2.7/site-packages/numpy/core/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c pomegranate/BayesianNetwork.c -o build/temp.macosx-10.10-intel-2.7/pomegranate/BayesianNetwork.o
    In file included from pomegranate/BayesianNetwork.c:256:
    In file included from /Users/code/Development/notebooks/env/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4:
    In file included from /Users/code/Development/notebooks/env/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:18:
    In file included from /Users/code/Development/notebooks/env/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1781:
    /Users/code/Development/notebooks/env/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: "Using deprecated NumPy API, disable it by "          "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
    #warning "Using deprecated NumPy API, disable it by " \
     ^
    pomegranate/BayesianNetwork.c:1842:8: error: 'inline' can only appear on functions
    static CYTHON_INLINE double (*__pyx_f_11pomegranate_5utils__log)(double); /*proto*/
           ^
    pomegranate/BayesianNetwork.c:210:27: note: expanded from macro 'CYTHON_INLINE'
        #define CYTHON_INLINE __inline__
                              ^
    1 warning and 1 error generated.
    error: command 'cc' failed with exit status 1

    ----------------------------------------
Command "/Users/code/Development/notebooks/env/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/mn/nf34qzvd11v3kd023kzyvzm00000gn/T/pip-build-O_xFKk/pomegranate/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/mn/nf34qzvd11v3kd023kzyvzm00000gn/T/pip-SHAxRC-record/install-record.txt --single-version-externally-managed --compile --install-headers /Users/code/Development/notebooks/env/include/site/python2.7/pomegranate" failed with error code 1 in /private/var/folders/mn/nf34qzvd11v3kd023kzyvzm00000gn/T/pip-build-O_xFKk/pomegranate

It seems like some form of C/Cython error is having it's toll. Am re-installing gcc to make sure that it's not due to something from python that is missing. I am using a mac, OSX Yosemite (10.10.5).

Anybody have any suggestions?

modifying an existing transition probability

Is there a way to modify a transition probability in a model manually?

In my specific use-case, I'd like to have this ability so I can enforce some constraints on the transitions.
Specifically, I would like to interleave a constraints enforcing method in-between single baum_welch iterations.

Thanks !
Lavi

pip install codec issues on Ubuntu 14.04

I've been having issues with installing pomegranate on Ubuntu 14.04, here's what pip install gives me:
...
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

---

Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip_build_root/pomegranate/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-bGdNcT-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/pomegranate
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in
load_entry_point('pip==1.5.4', 'console_scripts', 'pip')()
File "/usr/lib/python2.7/dist-packages/pip/init.py", line 235, in main
return command.main(cmd_args)
File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 104: ordinal not in range(128)
...

I understand that this is probably a C issue (which I don't understand much about) and I've tried the following:

1. Updating setuptools (sudo pip install --upgrade setuptools)
2. Trying to change the codec used by gcc to utf-8 instead of ascii (I don't think I succeeded doing this).
3. export LC_ALL=C (suggested by some posts on StackOverflow which were about trying to pip install numpy etc and giving a similar error)
4. Pulling pomegranate from github and doing python setup.py install

None of these helped - step 3 changed the error to this:
...
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

---

Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip_build_root/pomegranate/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-YKw7pt-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip_build_root/pomegranate
Storing debug log for failure in /home/narendra/.pip/pip.log
...

Here's my gcc version:
...
narendra@spokeyspikes:$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.8.4-2ubuntu114.04.1' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.1)
...

Any hints on how to go about handling this? Thanks so much in advance!

BayesianNetwork.forward_backward() fails on supply of soft evidence unless check_input set to False

Example - as per the BayesianNetwork example in pomegranate/examples/bayesnet_asia.py - where soft evidence is supplied (a DiscreteDistribution representing an 80:20 chance of bronchitis). The check_input part of the forward_backward method fails here, if the optional parameter check_input is set to False the execution can proceed.

'ClosureScope' object has no attribute 'scope_class'

Trying to install pomegranate (c8a246b) using 'sudo python setup.py install' on Ubuntu 14.10 with Python 2.7.10 and Anaconda 2.3.0. Using gcc and g++ versions 4.9.1.

guest@foo ~/installs/python/pomegranate (master) $ sudo python setup.py install
Compiling pomegranate/utils.pyx because it changed.
Compiling pomegranate/hmm.pyx because it changed.
Compiling pomegranate/FactorGraph.pyx because it changed.
Compiling pomegranate/base.pyx because it changed.
Compiling pomegranate/distributions.pyx because it changed.
Compiling pomegranate/BayesianNetwork.pyx because it changed.
Compiling pomegranate/fsm.pyx because it changed.
Compiling pomegranate/gmm.pyx because it changed.
Cythonizing pomegranate/BayesianNetwork.pyx
Cythonizing pomegranate/FactorGraph.pyx
Cythonizing pomegranate/base.pyx
Cythonizing pomegranate/distributions.pyx
Cythonizing pomegranate/fsm.pyx
warning: pomegranate/fsm.pyx:124:37: Index should be typed for more efficient access
Cythonizing pomegranate/gmm.pyx
Cythonizing pomegranate/hmm.pyx

Error compiling Cython file:
------------------------------------------------------------
...

        if verbose:
            print( "Total Training Improvement: {}".format( improvement ) )
        return improvement

    cdef double _train_baum_welch(self, list sequences, double stop_threshold, 
     ^
------------------------------------------------------------

pomegranate/hmm.pyx:2168:6: closures inside cdef functions not yet supported

Error compiling Cython file:
------------------------------------------------------------
...
                iteration += 1
                last_log_probability_sum = trained_log_probability_sum

                memset( expected_transitions, 0, m*m*sizeof(double) )

                parallel( delayed( self._baum_welch_summarize, check_pickle=False )(
                                                                      ^
------------------------------------------------------------

pomegranate/hmm.pyx:2213:71: Compiler crash in CreateClosureClasses

ModuleNode.body = StatListNode(hmm.pyx:7:0)
StatListNode.stats[23] = StatListNode(hmm.pyx:79:5)
StatListNode.stats[0] = CClassDefNode(hmm.pyx:79:5,
    as_name = u'HiddenMarkovModel',
    base_class_module = u'',
    base_class_name = u'Model',
    class_name = u'HiddenMarkovModel',
    doc = u'\n\tRepresents a Hidden Markov Model.\n\t',
    module_name = u'',
    visibility = u'private')
CClassDefNode.body = StatListNode(hmm.pyx:80:1)
StatListNode.stats[36] = CFuncDefNode(hmm.pyx:2168:6,
    args = [...]/11,
    doc = u'\n\t\tGiven a list of sequences, perform Baum-Welch iterative re-estimation on\n\t\tthe model parameters.\n\t\t\n\t\tIterates until the log of the "score" (total likelihood of all \n\t\tsequences) changes by less than stop_threshold. Returns the final log\n\t\tscore.\n\t\t\n\t\tAlways trains for at least min_iterations.\n\t\t',
    modifiers = [...]/0,
    needs_closure = True,
    visibility = u'private')
CFuncDefNode.body = StatListNode(hmm.pyx:2172:2,
    is_terminator = True)
StatListNode.stats[5] = WithStatNode(hmm.pyx:2201:7)
WithStatNode.body = TryFinallyStatNode(hmm.pyx:2201:7,
    preserve_exception = 1)
TryFinallyStatNode.body = TryExceptStatNode(hmm.pyx:2201:7,
    has_default_clause = 1)
TryExceptStatNode.body = StatListNode(hmm.pyx:2201:7)
StatListNode.stats[1] = StatListNode(hmm.pyx:2202:3)
StatListNode.stats[2] = WhileStatNode(hmm.pyx:2205:3)
WhileStatNode.body = StatListNode(hmm.pyx:2206:4)
StatListNode.stats[4] = ExprStatNode(hmm.pyx:2213:12)
ExprStatNode.expr = SimpleCallNode(hmm.pyx:2213:12,
    analysed = True,
    is_temp = 1,
    use_managed_ref = True)
SimpleCallNode.arg_tuple = TupleNode(hmm.pyx:2213:12,
    is_sequence_constructor = 1,
    is_temp = 1,
    result_is_used = True,
    use_managed_ref = True)
TupleNode.args[0] = GeneratorExpressionNode(hmm.pyx:2213:71,
    genexpr_name = u'genexpr4',
    is_temp = 1,
    name = u'genexpr',
    needs_closure = True,
    needs_self_code = True,
    pymethdef_cname = u'__pyx_mdef_11pomegranate_3hmm_17HiddenMarkovModel_17_train_baum_welch_1genexpr',
    result_is_used = True,
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "Visitor.py", line 170, in Cython.Compiler.Visitor.TreeVisitor._visit (Cython/Compiler/Visitor.c:4287)
  File "/usr/lib/python2.7/dist-packages/Cython/Compiler/ParseTreeTransforms.py", line 2364, in visit_LambdaNode
    self.create_class_from_scope(node.def_node, self.module_scope, node)
  File "/usr/lib/python2.7/dist-packages/Cython/Compiler/ParseTreeTransforms.py", line 2342, in create_class_from_scope
    type=cscope.scope_class.type,
AttributeError: 'ClosureScope' object has no attribute 'scope_class'
Traceback (most recent call last):
  File "setup.py", line 36, in <module>
    extensions = cythonize( extensions )
  File "/usr/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 798, in cythonize
    cythonize_one(*args[1:])
  File "/usr/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 915, in cythonize_one
    raise CompileError(None, pyx_file)
Cython.Compiler.Errors.CompileError: pomegranate/hmm.pyx

MixtureDistribution.from_sample

MixtureDistribution.from_sample's doc mentions that it is unimplemented, but it seems to be implemented to me :)

Pip won't install, possible typo

I think you have a typo in the version of joblib you are requiring. In the setup it's looking for 0.90b4, but all pip can find is 0.9.0b4 and thus I think this is the reason it's failing to install for me.

ImportError: no module named utils

(Hi. I hope this is appropriate here, otherwise let me know where to post this, I'm not an expert)
After installation in debian stable (by sudo pip install pomegranate), I get the following error while importing pomegranate in ipython:

In [1]: import pomegranate

ImportError                               Traceback (most recent call last)
<ipython-input-2-f115d95d8f94> in <module>()
----> 1 import pomegranate
/usr/local/lib/python2.7/dist-packages/pomegranate/__init__.py in <module>()
     36     pyximport.install()
     37 
---> 38 from .hmm import *
     39 from .BayesianNetwork import *
     40 from .FactorGraph import *
/home/fpdx/pomegranate/hmm.pyx in init pomegranate.hmm (pomegranate/hmm.c:53433)()
/home/fpdx/pomegranate/distributions.pxd in init pomegranate.base (pomegranate/base.c:22885)()
/home/fpdx/pomegranate/distributions.pyx in init pomegranate.distributions (pomegranate/distributions.c:65203)()

ImportError: No module named utils

Do you know what is it missing?

The end of the installation gave this :

Installing cython script to /usr/local/bin 
    Installing cygdb script to /usr/local/bin`
    Installing cythonize script to /usr/local/bin
  Found existing installation: joblib 0.8.3
    Not uninstalling joblib at /usr/lib/python2.7/dist-packages, owned by OS
Successfully installed pomegranate cython joblib networkx
Cleaning up...

To check the installation and versions:

$ pip show pomegranate

---
Name: pomegranate
Version: 0.3.7
Location: /usr/local/lib/python2.7/dist-packages
Requires: cython, numpy, joblib, networkx

$ pip show joblib

---
Name: joblib
Version: 0.9.4
Location: /usr/local/lib/python2.7/dist-packages
Requires: 

$ pip show cython

---
Name: Cython
Version: 0.23.4
Location: /usr/local/lib/python2.7/dist-packages
Requires: 

$ pip show networkx

---
Name: networkx
Version: 1.11
Location: /usr/local/lib/python2.7/dist-packages
Requires: decorator

$ pip show numpy

---
Name: numpy
Version: 1.8.2
Location: /usr/lib/python2.7/dist-packages
Requires: 

As a side note, if I repeat import pomegranate right after this error multiple times, I get the error with highlighted line changes to
---> 39 from .BayesianNetwork import *
next import pomegranate:
---> 44 from .gmm import *
next import pomegranate:
---> 45 from .NaiveBayes import *
and next import pomegranate doesn't give an error. But then ipython cannot stand it:

In [6]: pomegranate.NormalDistribution(1,2)
Segmentation fault

Exception AttributeError in training a HMM (code was working with yahmm)

Here's a reduced example of the general setup I had been using with yahmm (the model wasn't converging to the right parameters - I'll post about that later - but it was running without errors):

test_pomegranate.txt

Here's the error I get when I try to run this:
...
Exception AttributeError: 'exit' in 'pomegranate.hmm.HiddenMarkovModel._train_loop' ignored
Total Training Improvement: 0.0
...

Any suggestions? Thanks a lot in advance!

BayesNet forward_backward returns incorrect results

The backward step in the sum-product algorithm is producing incorrect results. I'm on the case.

python 3.4/3.5 has issues on unit tests

Bayesian network unit tests seem to fail randomly in Python 3.4/3.5, but are always successful on Python 2.7. Given that I only recently started unit testing in Python 3.4/3.5, it is unclear if this is a new issue presented in the distribution speedups, or an issue always present.

Error in `os` under Python3.5

It looks like pomegranate is not compatible with Python3.5's os.environ:

>>> import pomegranate as pom
...
    elif os.name == 'posix':
-->       if os.environ.has_key('CFLAGS'):
...
AttributeError: '_Environ' object has no attribute 'has_key'

write a wiki

I'm leaving this to remind myself that other people aren't me, and probably need to know what's in the package in an easier format than scanning the code.

Fail to update hmm.graph after training

The hmm's model.graph's edge weights are not being updated after training.

'ConditionalDiscreteDistribution' is not defined

I'm trying to use ConditionalDiscreteDistribution according to the examples but getting this error. ConditionalDiscreteDistribution seems to be missing in the repo as well when I was searching for it. Am I missing something?

Exception in _train_once_baum_welch with MultivariateGaussianDistribution

I'm having a problem when using MultivariateGaussianDistribution in a HMM. My HMM has 10 states and is used with sequences that have 43 features. My covariance matrix is therefore 43x43 and diagonal. I do pass in the diagonal=True flag when training, but the described problem also occurs if it is set to False. When training with Baum-Welch, the library will log with the following exception:

Exception TypeError: "object of type 'NoneType' has no len()" in 'pomegranate.hmm.HiddenMarkovModel._train_once_baum_welch' ignored

This happens in the first round and the training improvement will then be 0.0. Naturally, the algorithm will then terminate since the threshold is reached.

I've tried to figure out exactly what is going on, but I don't have a lot of experience with Cython and wasn't able to figure out where the exception is raised.

It would be great if someone has ideas what could cause this. If someone has pointers on how to find the source of the exception I'm more than glad to do some debugging myself, too.

pip

I am currently having issues getting pomgranate to be pip installable, stemming from the use of multiple cython files. If anyone knows the fix, please submit a working pull request and I will send you a picture of a pomegranate and other assorted fruit of varying deliciousness.

Could you be more specific? I looked at your setup.py but it seems OK.

calling MultivariateGaussianDistribution.to_json() fails

import pomegranate
d=pomegranate.MultivariateGaussianDistribution([0,0],[[1.0],[0,1]])
d.to_json()

causes

TypeError: array([0, 0]) is not JSON serializable

It seems the parameters are stored as numpy arrays and these are not serializable
Perhaps you could hold them as lists and convert to numpy array only when you need to use them?

(BTW, I'm pretty sure this a a bug that's unrelated to my recent tinkering with the to_json() code. I checked on a fresh install of pomegranate from the current source).

error: command 'gcc' failed with exit status 1

I tried searching for postings on this in relation to cython, with no luck. I get this error when using anaconda python or otherwise. I still receive it after upgrading numpy and cython. Any ideas appreciated:

gcc -fno-strict-aliasing -I/Users/bob/anaconda/include -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -arch x86_64 -I/Users/bob/anaconda/lib/python2.7/site-packages/numpy/core/include -I/Users/bob/anaconda/include/python2.7 -c pomegranate/base.c -o build/temp.macosx-10.5-x86_64-2.7/pomegranate/base.o
In file included from pomegranate/base.c:239:
In file included from /Users/bob/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /Users/bob/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:
In file included from /Users/bob/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:
/Users/bob/anaconda/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning:
"Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION"
[-W#warnings]

warning "Using deprecated NumPy API, disable it by " \

^
pomegranate/base.c:1764:8: error: 'inline' can only appear on functions
static CYTHON_INLINE double (___pyx_f_11pomegranate_5utils__log)(double); /_proto*/
^
pomegranate/base.c:174:27: note: expanded from macro 'CYTHON_INLINE'
#define CYTHON_INLINE inline
^
1 warning and 1 error generated.
error: command 'gcc' failed with exit status 1

Loading HMM with from_json() results in: [RuntimeWarning: divide by zero encountered in log]

For my project i store the HMMs on disks and load them when necessary. While doing this i found a bug, i think.

Principal Code:
Storing

      jsonFile = hmm.to_json()
      json.dump(jsonFile, file)

Loading

    jsonFile = json.load(file)
    hmm = HiddenMarkovModel.from_json(jsonFile)

So far so good. It is working and generating a HiddenMarkovModel. The only problem is the message in the headline. (RuntimeWarning: divide by zero encountered in log) And i get this for every stored Model. There is either an operation error on my side or the from_json() method is something not handling well.

For further tests i included on of the hmms:
HMM_0.txt

Tutorial 1: Error in Monty's conditional probability table

Not directly a bug, but...

https://github.com/jmschrei/pomegranate/blob/master/tutorials/Tutorial_1_Distributions.ipynb

The line "A', 'B', 'A', 0.5" of Monty's conditional probability table should have probability 0.0 because Monty cannot open the door the guest has chosen.

support for missing observations (in HMM) ?

Hi.
Does pomegranate support missing observations?
I'd like to do something like:

# make a series with a switch point 
obs1 = numpy.random.normal(0,2,100)
obs2 = numpy.random.normal(1,2,100)
obs = numpy.concatenate([obs1, obs2])

# sprinkle missing values
obs[0::10] = numpy.nan

# model = generate a pomegranate.HiddenMarkovModel with two states

model.train([obs])

If not, what changes would be needed in order to support this?

Proposal: Make HMM conform to deepcopy

Scenario:

In my project i need to concatenate (sometimes) the same model once, twice or multiple times. To achieve this, i used to save it as json and reload it to get another instance. This was necessary to get two models, the original one and the concatenated one for comparison.

This comes with two problems:

1. At the moment a bug while saving/loading is messing with my HMM edge values (explained in #72)
2. It is a workaround.

Sadly the deepcopy function is not working properly. Though it doesn't throw an error the new HMM instance is still broken.

When i use the Frontpage example for the HMM, the following is happening:

-0.505202786679
Exception TypeError: "object of type 'NoneType' has no len()" in 'pomegranate.hmm.HiddenMarkovModel.log_probability' ignored
0.0

with

seq = 'ACT'
hmm = initHMM('Test')
print(hmm.log_probability(seq))
# Deep copy
hmm2 = copy.deepcopy(hmm)
print(hmm2.log_probability(seq))

HMM - After saving a Model to JSON/Load it from JSON the Edge values have changed

I use a HMM with IndependentComponent Distributions and after the process of saving the json and loading it afterwards, the classifications results differed. The Output values changed like in the following log:

Before Saving

INFO 2016-02-03 10:55:00,959 EM_Classification[25812] Classification for Activity S0A0
INFO 2016-02-03 10:55:00,959 EM_Classification[25812] Device: DT_Apple Watch_DP_Unknown_DA_Unknown
INFO 2016-02-03 10:55:00,959 EM_Classification[25812] All Classification Probabilities:
INFO 2016-02-03 10:55:00,959 EM_Classification[25812] Right Lunge: -inf
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Burpee: -inf
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Raise Watch: 466.344238952
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Squat: 74.1050046185
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Left Lunge: -640.643100476
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Jump: -273.247585414
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Jumping Jack: 79.4273948526
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Straight Leg Lever: -9594.14306721
INFO 2016-02-03 10:55:00,960 EM_Classification[25812] Pushup: -10680.2996986
INFO 2016-02-03 10:55:00,961 EM_Classification[25812] High Jump: -1304.46328608
INFO 2016-02-03 10:55:00,961 EM_Classification[25812] Lower Watch: 515.155386954
INFO 2016-02-03 10:55:00,961 EM_Classification[25812] Situp: -inf
INFO 2016-02-03 10:55:00,966 EM_Classification[25812] 

After Loading

INFO 2016-02-03 10:55:11,501 EM_Classification[25812] Classification for Activity S0A0
INFO 2016-02-03 10:55:11,501 EM_Classification[25812] Device: DT_Apple Watch_DP_Unknown_DA_Unknown
INFO 2016-02-03 10:55:11,501 EM_Classification[25812] All Classification Probabilities:
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Right Lunge: -8.71385277776e+33
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Burpee: -2.94670031904e+33
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Raise Watch: -6.93503524546e+32
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Squat: -3.96725476297e+33
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Left Lunge: -3.22029792202e+34
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Jump: -3.26391445429e+32
INFO 2016-02-03 10:55:11,502 EM_Classification[25812] Jumping Jack: -1.48371979441e+33
INFO 2016-02-03 10:55:11,503 EM_Classification[25812] Straight Leg Lever: -2.03141513145e+34
INFO 2016-02-03 10:55:11,503 EM_Classification[25812] Pushup: -5.61842917876e+34
INFO 2016-02-03 10:55:11,503 EM_Classification[25812] Lower Watch: -8.28474371382e+32
INFO 2016-02-03 10:55:11,503 EM_Classification[25812] High Jump: -3.30337906051e+33
INFO 2016-02-03 10:55:11,503 EM_Classification[25812] Situp: -9.92683566223e+32
INFO 2016-02-03 10:55:11,508 EM_Classification[25812] 

After looking into the JSON files of one of the HMMs i found the following, on the first look the Independent Component Dists contained the same values as prior to the process. But the edge values changed suddenly. For further comparison i attached two TXT Files with the JSON representation of a HMM model. The first json represents the Model before it was changed by the from_json() method. Second JSON represents the model afterwards.

From my observation the edge values are changed in the from_json() method or submethods. I considered that the change may occur when saving the model (via to_json()) but my oberservation say otherwise. One interesting thing is, the model is only changing once. With further save/loading processes the output of the classification stayed the same.

I think that the Disttributions have an influence on your edge probabilities, because they only change once and the Distribution values never change. So the Edge calculation is influenced by a fixed factor which could be the Distribution.

HMMPriorLoading.txt
HMMAfterLoading.txt

float division by zero error at BayesianNetwork.bake()

Hi

I'm trying to create a simple Bayesian Network with two states, but I'm confused with the way ConditionalProbabilityTable works.

# The hardware works correcty 99 out 100 times
hardware = DiscreteDistribution( { 'w': 0.99, 'nw': 0.01 } )


# The editor works if the hardware works
editor = ConditionalProbabilityTable(
        [['w', 'w', 1],
         ['nw', 'w', 0],
         ['w', 'nw', 0],
         ['nw', 'nw', 1]],[hardware])

s1 = State( hardware, name="hardware" )
s2 = State( editor,name="editor")

# Create the Bayesian network object with a useful name
network = BayesianNetwork( "Freds LISP Problem" )


network.add_nodes([s1, s2])

network.add_edge(s1, s2)
network.bake()

This runs without error, whereas if I modify the CPT to have different keys, it ends up in float by division error at network.bake()

# The editor works if the hardware works
editor = ConditionalProbabilityTable(
        [['w', 'work', 1],
         ['nw', 'work', 0],
         ['w', 'notwork', 0],
         ['nw', 'notwork', 1]],[hardware])

Two different states can take up different sets of values, but the CPT doesnt seems to accept it

Linear Gaussian Bayesian Networks

This has been a big issue for a while in that I don't support continuous output conditional probabilities or their Bayesian network counterparts.

Question: Sampling from conditional probability

Hello,
I don't know if this is the appropriate place to ask this, please tell me if it isn't.
For my master thesis I would like estimate conditional probabilities of multiple random variables which can be both discrete and continuous (mixture of continuous and discrete variables). I think this could be done using pomegranate by doing density estimation on this mixture and on the joint distribution of the evidence variables. The result would a fraction of 2 mixture of distributions, but not a closed form distribution. Thus, to sample from the conditional distribution I would have to resort rejection sampling (or an optimised MCMC method). Is this correct?

Error: pomegranate.fsm.FiniteStateMachine' object has no attribute 'add_states from

Getting this error while trying demo code from README.md

I am doing it wrong ?

Thanks