tianjg/adversarialregression

Adversarial_regression

Generative Adversarial Networks for Non-Linear Regression: Theory and Assessment

Master Thesis: Yoann Boget
Supervisor at SLAC (Stanford University: Dr. Micheal Kagan
Supervisor at University of Neuchâtel: Dr. Clément Chevalier
https://arxiv.org/abs/1910.09106
Grade: 5.5/6 (Very good)

Abstract

Adversarial Regression is a proposition to perform high dimensional non-linear regression with uncertainty estimation. It uses Conditional Generative Adversarial Network to obtain an estimate of the full predictive distribution for a new observation.

Generative Adversarial Networks (GAN) are implicit generative models which produce samples from a distribution approximating the distribution of the data. The conditional version of it can be expressed as follow:

where D and G are real-valuated functions, x and y respectively the explained and explanatory variables, and z a noise vector from a known distribution, typically the standard normal. An approximated solution can be found by training simultaneously two neural networks to model D and G. After training, we have that G(z, y) approximate p(x, y). By fixing y, we have G(z|y) approximating p(x|y). By sampling z, we can therefore obtain samples following approximately p(x|y), which is the predictive distribution of x for a new observation y.

We ran experiments to test various loss functions, data distributions, sample sizes, and dimensions of the noise vector. Even if we observed differences, no set of hyperparameters consistently outperformed the others. The quality of CGAN for regression relies on fine-tuned hyperparameters depending on the task rather than on fixed values. From a broader perspective, the results show that adversarial regressions are promising methods to perform uncertainty estimation for high dimensional non-linear regression.