victorzxy/function-autoencoder

Investigative project for my CST Part III Probabilistic Machine Learning (LE49) module

Function Autoencoders: A Neural Network Approach to Gaussian Processes

Investigative project of my CST Part III Probabilistic Machine Learning (LE49) module

Report here

Abstract

Gaussian processes (GPs) are data-efficient and flexible probabilistic methods that learn distributions of functions based on given priors. However, GPs suffer from unscalability as they become very computationally expensive on large datasets, and choosing the appropriate priors for GPs can be nontrivial. In this project, I investigated a neural network (NN) alternative to GPs, and introduced the function autoencoders that preserve GPs’ own advantages and avoid their weaknesses with NNs’ benefits. I tested the performance of the various function autoencoders on a 1-dimensional function regression task, and compared and analysed their results. The trained function autoencoder models indeed have the ability to learn distributions over random functions, and performed decently on the selected task. Moreover, the function autoencoders demonstrate a great potential for further improvements.