dpiras/gmm-mi

Estimation of mutual information (MI) distribution with Gaussian mixture models (GMMs)

gmm-mi's Issues

Restructure code to perform fitting and estimation separately

See #2.

Extend to higher-dimensional variables

So far, GMM-MI can only be applied to unidimensional variables; we want to extend it to the multivariate case. It should not be too much work, since we showed that we can extend to higher dimensionalities with conditional MI.

We need to choose if we can re-use the methods currently implemented to estimate MI between unidimensional variables, or whether it is cleaner to create three separate methods. Once implemented, we plan to test this against e.g. work done in Sui et al. (2023).

Create method for contour plots

To better visualise the fitted model of the joint pdf, especially in higher dimensions when the conditional MI is computed, it would be nice to use something like chainconsumer to use the original samples and samples from the fitted GMM to obtain contour plots to compare.

Add warning when categorical fit and quadrature method required

As in the title; we only implemented MC integration when estimating categorical-continuous MI.

Also remove reference to loss curves when doing the categorical fit (or check they are indeed stored).

Add different units

Should be a simple tweak at the end, based on the required base of the logarithm (nats, bits)

Store all metrics

When fitting a model, a single metric (validation loss, AIC or BIC) is used to select the number of components. However it might be nice to also store the value of the BIC and AIC (if using validation) so that one does not have to re-fit the model to get the other values of the metrics.

More flexibility in the input shape

So far, only arrays with shape (n_samples, 2) can be passed in the continuous case. This can be annoying if I have the samples from the two variables as separate arrays, since one needs to manually stack them. It should be easy to add the possibility of passing either a 2-D array, or two 1-D arrays.