statml/zhusuan

A Library for Generative Models

ZhuSuan

ZhuSuan is a python library for Generative Models, built upon Tensorflow. Unlike existing deep learning libraries, which are mainly designed for supervised tasks, ZhuSuan is featured for its deep root into Bayesian Inference, thus supporting various kinds of generative models: both the traditional hierarchical Bayesian models and recent deep generative models.

With ZhuSuan, users can enjoy powerful fitting and multi-GPU training of deep learning, while at the same time they can use generative models to model the complex world, exploit unlabeled data and deal with uncertainty by performing principled Bayesian inference.

Supported Inference

(Stochastic) Variational Inference (VI & SVI)

• Kinds of variational posteriors we support:

  • Mean-field posterior: Fully-factorized.
  • Structured posterior: With user specified dependencies.

• Variational objectives we support:

  • SGVB: Stochastic gradient variational Bayes
  • IWAE: Importance weighted objectives
  • NVIL: Score function estimator with variance reduction
  • VIMCO: Multi-sample score function estimator with variance reduction

Adaptive Importance Sampling

• Reweighted Wake-sleep (RWS): With user specified adaptive proposal.

Markov Chain Monte Carlo (MCMC)

• Hamiltonian Monte Carlo (HMC): With step size and mass adaptation.

Installation

ZhuSuan is still under development. Before the first stable release (1.0), please clone the repository and run

pip install .

in the main directory. This will install ZhuSuan and its dependencies automatically.

If you are developing ZhuSuan, you may want to install in an "editable" or "develop" mode. See the Developments section below.

Documentation

Documentation is available online: zhusuan.readthedocs.io

Examples

To run the provided examples, you may need extra dependencies to be installed. This can be done by

pip install ".[examples]"

• Gaussian: HMC
• Toy 2D Intractable Posterior: SGVB
• Beyesian Neural Networks: SGVB
• Variational Autoencoder (VAE): SGVB, IWAE
• Convolutional VAE: SGVB
• Semi-supervised VAE (Kingma, 2014): SGVB, RWS
• Deep Sigmoid Belief Networks RWS, VIMCO
• Logistic Normal Topic Model: HMC

Developments

To install ZhuSuan in an "editable" or "develop" mode, run

pip install -e .

in the main directory. This installation is removable by

pip uninstall zhusuan

Additional dependencies for developments can be installed by

pip install ".[dev]"

Tests

This command will run automatic tests in the main directory.

python -m unittest discover -v

Test Coverage

After running tests, to ensure test coverage over the developments, run

coverage report --include="zhusuan/*"

PEP8 Code Style Check

We follow PEP8 python code style. To check, in the main directory, run

pep8 .

Docs

Docs are written under docs/ directory as RestructuredText (.rst) files. index.rst is the main page. A Tutorial on RestructuredText can be found here.

API References are automatically generated by Sphinx. They are under doc/api/ directory and should be regenerated each time when any code changes:

make api

To compile docs into webpages, Run

make html

under docs/ directory. The generated webpages are in docs/_build and can be viewed with browsers.