modifying an existing transition probability

[jmschrei]

modifying an existing transition probability

[jmschrei]

Currently there isn't. I'm using a three vector approach to sparse matrices, but each vector is a private attribute right now. Can you describe what type of constraints you'd like to put in? You can tie edges and add edge inertia during training, if either of those help.

[shpigi]

I'd like to learn an autoregressive (AR)HMM. In this model, there is an inner hidden variable that is connected to itself at next time point and an outer hidden variable. The outer hidden variable is connected to itself in the next time point and to the observation.
This model can be simulated and used for inference in a regular HMM by by creating a single hidden variable whose state space is the Cartesian product of the states of the original hidden variables, plus some tying of the distributions.
To learn this ARHMM, the hmm learning procedure needs a modification. I run the Baum-Welch procedure with the Cartesian-product model, use the returned expected_transitions matrix (which I made Baum-Welch return just like forward-backward does) to calculate the sufficient statistics of the AR model and turn those back into the transition probabilities of the Cartesian-product HMM. This is the point where I need to update the model's transition probabilities.

A possible work-around would be to re-build the model from scratch every iteration but I would have thought that others may find such a feature useful.

Another use-case for this is a program for what-if scenario analysis where the user may want to make modifications to a model and re-run it. The same workaround (regeneration the model) would apply.

[shpigi]

(edited my previous comment to clarify it a bit)

[jmschrei]

Autoregressive HMMs are super cool, and I would like to add more functionality to both the HMM section and the Bayes net section to make them (and other wild HMM models) easier to build/train/use. However, time is a major limitation in my life right now--I haven't even had time to flesh out factor graphs, which was my goal for this quarter.

Rebuilding the model from scratch is a currently viable solution, though very hacky and inefficient. I agree there should be a better way; and adding it shouldn't be too difficult. I'll add in a method which takes in either a dense or sparse matrix and calculates a new internal transition matrix from that.

If you need something very soon, the model regeneration technique shouldn't be too difficult to code (just computationally intensive) and would be something like the following:

from pomegranate import HiddenMarkovModel

distributions = [ list of distributions, not states ]
transition_matrix = [ transitions here ]

for i in xrange( max_iterations ):
    model = HiddenMarkovModel.from_matrix( transition_matrix, distributions, starts=[ probability of starting in each state], ends=[ probability of ending in each state ], state_names = [ state names ] )
    model.bake()
    model.train( training_data, max_iterations=1 )

    expected_transitions = model.forward_backward( training_data )[0]
    transition_matrix = * AR calculation magic *

** edited code slightly **

Let me know if this helps. When I have time I'll add in the new function.

[shpigi]

Thanks.
In your code snippet you'd need to pass distributions to from_matrix() (instead of states).
In my use case, I'd also need to do some distribution tying but I guess I can do it by properly arranging the distributions in the distributions vector.
Question - in this code snippet I'd be running forward-backward twice per iteration (once within Baum-Welch). I imagine that letting Baum-Welch return the expected_transitions matrix would eliminate the need and significantly shorten each iteration. What do you think?

[jmschrei]

Whoops, my bad. I actually went and checked if it took distributions or states, then updated the distributions at the top, but forgot to in the actual call.

As long as you know which distributions you want to tie, remembering which indices refer to the same distribution shouldn't be too bad, and then just passing in one distribution object for each index in the future.

There is a cost, but you don't want to just return the matrix Baum-Welch calculates, because you're updating your parameters using it causing it to be out of date. You're replicating effort because the forward_backward matrix you calculate in iteration 3 should be the same as the Baum-Welch matrix in iteration 4.

Though, now that I think about it, if you're externally regularizing parameters based on autoregression that isn't even true anymore. The Baum-Welch matrix is going to be different (I think) than the previous iterations forward_backward, because you've changed parameters between the two calculation.

I would start off with the possible duplication of effort as a proof of concept, and then see if the forward_backward matrix and the Baum-Welch are the same.

[shpigi]

You're right, both about the out-of date Baum Welch and the changes caused by the external transition probability updates invalidating the forward-backward result.
Thanks.

[jonnor]

Is there a bette way to achieve this now? Or a complete example that demonstrates the use of modifying the transition matrix to enforce some simple constraints (like a certain transition being impossible, ie probability of 0.0)?

[jonnor]

Looking into the code, it seems the on_epoch_end callback is called on each iteration of the E-M / Baum-Welch learning loop. So maybe it is possible to hook that up to a function that manipulates the transition matrix to enforce some constraints? This would avoid having to re-create a new model and to make a custom training loop.