slaypni/universal-divergence

A Python module for estimating divergence between two sets of samples.

universal-divergence

universal-divergence is a Python module for estimating divergence of two sets of samples generated from the two underlying distributions. The theory of the estimator is based on a paper written by Q.Wang et al [1].

Install

pip install universal-divergence

Example

from __future__ import print_function

import numpy as np
from universal_divergence import estimate

mean = [0, 0]
cov = [[1, 0], [0, 10]]
x = np.random.multivariate_normal(mean, cov, 100)
y = np.random.multivariate_normal(mean, cov, 100)
print(estimate(x, y))  # will be close to 0.0

mean2 = [10, 0]
cov2 = [[5, 0], [0, 5]]
z = np.random.multivariate_normal(mean2, cov2, 100)
print(estimate(x, z))  # will be bigger than 0.0

References

[1]	Qing Wang, Sanjeev R. Kulkarni, and Sergio Verdú. "Divergence estimation for multidimensional densities via k-nearest-neighbor distances." Information Theory, IEEE Transactions on 55.5 (2009): 2392-2405.

universal-divergence's Issues

negative divergence value

hi thanks for the repository...
I was using the code to calculate the divergence between gaussian distribution but I was getting negative divergences. Mean vectors for both gaussians was a 2 dimensional vector, each entry was chosen randomly between (0,1) and covariance matrix was an identity matrix.
Calculating Kl divergence by using K = NONE gives negative values.