stephantul/somber

Recursive Self-Organizing Map/Neural Gas.

somber's Issues

Batching for recurrent SOMS

All recurrent SOMs would benefit from some kind of batching scheme. I currently have batched code, but it doesn't work. My hunch is that batching the recurrent/recursive/merging SOM is non-trivial because there is no way of making sure the concurrently processed batches converge on the same BMU.

Because we take the mean over batches, this means that updates will generally end up being more noisy as the batch size increases.

Two starting ideas:

1. A possible solution could be to initialize the context Weights and weights of the SOM to some value which allows for convergence (e.g. the weights are skewed in such a way that the concurrently processed batches all go in the same direction), but this entails setting the parameters in advance, which kind of defeats any purpose the SOM has.

2. A second solution would be to slowly increase the batch size, allowing the SOM to converge to something stable first, and then increasing batch size.

Add PLSOM2

The PLSOM2 is an interesting candidate for addition to SOMBER, for the following reasons:

• we already have PLSOM
• It is supposed to improve PLSOM by making it less sensitive to outliers

It would require changing the updates routine and parameter update routine in the PLSOM.

Dependabot couldn't authenticate with https://pypi.python.org/simple/

Dependabot couldn't authenticate with https://pypi.python.org/simple/.

You can provide authentication details in your Dependabot dashboard by clicking into the account menu (in the top right) and selecting 'Config variables'.

View the update logs.

Add Gamma som

The Merge som is working, so we perhaps could implement the Gamma Som, which uses a generalization of the Merge Som context function (see here (behind a paywall unfortunately)).

The Merge Som Best Matching Unit (BMU) calculation takes into account the BMU at the previous time step, together with its context weight, as follows:

 context = (1 - self.beta) * self.weights[prev_bmu] + self.beta * self.context_weights[prev_bmu]

As context weights reflect the previous activation, you can use a merge som to "step back in time"

Generalizing over this, the Gamma Som defines a BMU function which takes into account the k previous BMUs explicitly (so the Merge Som can be defined as a Gamma SOM with k = 1). This makes the Gamma Som computationally more expensive than the Merge Som, but also more powerful.

Adding the Gamma Som will require relatively few changes, but I don't have any need for it currently.