djsutherland/opt-mmd

Learning kernels to maximize the power of MMD tests

Code for the paper "Generative Models and Model Criticism via Optimized Maximum Mean Discrepancy" (arXiv:1611.04488; published at ICLR 2017), by Dougal J. Sutherland (@dougalsutherland), Hsiao-Yu Tung, Heiko Strathmann (@karlnapf), Soumyajit De (@lambday), Aaditya Ramdas, Alex Smola, and Arthur Gretton.

• Implementations of the variance estimator are in Theano in two_sample/mmd.py and in Tensorflow in gan/mmd.py.
• General code for learning kernels for a fixed two-sample test, with Theano, is in two_sample.
• Code for the GAN variants, using TensorFlow, is in gan.
• Code for the efficient permutation test described in Section 3 is in the 6.0 release of Shogun; look under shogun/src/shogun/statistical_testing. An example of using it in the Python API is in two_sample/mmd_test.py.

This code is under a BSD license, but if you use it, please cite the paper.

opt-mmd's Issues

tmmd not maximizing the test power

Hi,
Thanks for sharing. In the tmmd_model.py file, the optimizer seems just trys to minimize the ratio-loss, which should be the approximated test power mentioned in the paper. Shouldn't we increase the test power? I can't find any "-" there. Do I miss anything?

Python package tables is required but not automatically installed

Issue: python package tables is required by pandas, but not automatically installed.

How to reproduce:

python fixed_run.py -n 500 --blobs 1

Error message:

HDFStore requires PyTables, "No module named tables" problem importing

Fix:
pip install tables fixed the problem for me.
Add requirement tables to requirements.txt

System:
MacOS 10.11., python 2.7.12

Missing coefficient for cross term in MMD

Apologies if I'm missing something obvious, but the estimate defined in equation (3) of https://www.jmlr.org/papers/volume13/gretton12a/gretton12a.pdf looks slightly different to the MMD estimate implemented here.

The first term makes sense to me, and subtracting m accounts for the summation over i != j (since diagonal of RBF kernel will add to m). Likewise for second term.
However the third term seems to be missing a factor 1 / (m*n).

opt-mmd/two_sample/mmd.py

Lines 44 to 49 in 5c02a92

	m = K_XX.shape[0]
	n = K_YY.shape[0]
	mmd2 = ((K_XX.sum() - m) / (m * (m - 1))
	+ (K_YY.sum() - n) / (n * (n - 1))
	- 2 * K_XY.mean())

Is this implementation intended to match the estimate linked above?

Cheers